Jul 02 23:22:25.779324 ip-10-0-130-160 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:22:26.202172 ip-10-0-130-160 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:26.202172 ip-10-0-130-160 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:22:26.202172 ip-10-0-130-160 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:26.202172 ip-10-0-130-160 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:22:26.202172 ip-10-0-130-160 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:22:26.203228 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.203136 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:22:26.207772 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207752 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.207772 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207772 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207776 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207779 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207782 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207784 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207787 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207790 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207792 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207795 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207797 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207800 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207802 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207805 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207807 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207810 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207812 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207815 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207817 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207820 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207822 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.207847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207824 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207827 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207829 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207833 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207836 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207838 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207843 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207847 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207850 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207854 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207859 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207863 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207866 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207868 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207871 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207874 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207877 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207879 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207881 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.208343 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207884 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207886 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207889 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207891 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207894 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207896 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207899 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207901 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207904 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207908 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207910 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207913 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207915 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207917 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207920 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207922 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207925 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207929 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207931 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207934 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.208811 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207936 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207939 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207941 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207944 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207946 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207949 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207952 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207954 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207957 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207961 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207963 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207965 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207968 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207971 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207975 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207978 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207981 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207984 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207987 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.209312 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207989 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207992 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207995 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.207998 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208001 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208019 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208022 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208025 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208029 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208033 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208036 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208039 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208042 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208045 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208048 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208051 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208054 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208057 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208060 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.208063 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.209763 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209742 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209750 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209754 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209757 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209760 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209763 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209765 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209768 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209771 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209775 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209778 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209781 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209783 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209786 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209789 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209791 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209794 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209796 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209799 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209802 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.210292 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209804 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209807 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209810 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209814 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209817 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209820 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209823 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209825 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209828 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209830 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209832 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209835 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209837 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209839 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209842 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209845 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209848 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209850 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209852 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209855 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.210769 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209857 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209860 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209863 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209865 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209867 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209870 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209872 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209874 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209877 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209880 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209883 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209885 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209888 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209891 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209893 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209895 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209898 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209901 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209903 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209905 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.211305 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209908 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209910 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209912 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209915 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209917 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209919 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209923 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209925 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209928 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209930 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209932 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209935 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209937 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209941 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209944 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209947 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209951 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209955 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209957 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.211795 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209960 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209962 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209965 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209967 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209970 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209972 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209974 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209977 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209979 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209981 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209985 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209988 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209990 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209993 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209995 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.209998 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210000 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210020 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210023 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210027 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.212286 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210420 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210427 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210430 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210433 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210435 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210438 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210441 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210443 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210445 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210448 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210450 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210453 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210455 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210458 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210460 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210463 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210465 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210467 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210470 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.212762 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210472 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210474 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210477 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210479 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210483 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210485 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210488 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210490 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210493 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210496 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210498 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210500 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210504 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210506 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210509 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210511 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210514 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210517 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210519 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210522 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.213231 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210524 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210526 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210529 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210531 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210533 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210536 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210538 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210541 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210543 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210545 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210547 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210550 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210552 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210554 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210557 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210560 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210562 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210565 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210567 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.213707 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210570 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210572 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210575 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210577 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210580 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210582 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210585 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210587 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210589 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210592 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210594 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210599 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210602 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210605 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210608 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210611 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210613 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210616 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210619 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210621 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.214169 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210624 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210626 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210629 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210631 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210634 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210636 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210638 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210641 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210644 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210646 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210648 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210651 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210655 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210659 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210662 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210664 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210667 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210670 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210673 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210675 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.214645 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.210677 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.210751 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.210760 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211589 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211595 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211599 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211602 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211607 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211611 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211614 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211617 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211621 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211624 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211627 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211630 2562 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211633 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211636 2562 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211639 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211642 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211647 2562 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211649 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211653 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211656 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211659 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:22:26.215195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211663 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211667 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211670 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211674 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211677 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211680 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211683 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211686 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211689 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211693 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211696 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211699 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211702 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211705 2562 flags.go:64] FLAG: --enable-server="true" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211708 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211712 2562 flags.go:64] FLAG: --event-burst="100" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211715 2562 flags.go:64] FLAG: --event-qps="50" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211717 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211720 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211723 2562 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211727 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211730 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211733 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211736 2562 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211739 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:22:26.215765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211742 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211750 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211753 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211757 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211760 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211763 2562 flags.go:64] FLAG: --feature-gates="" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211767 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211770 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211773 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211776 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211779 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211782 2562 flags.go:64] FLAG: --help="false" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211785 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211788 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211791 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211794 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211798 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211802 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211805 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211808 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211811 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211814 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211816 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211819 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:22:26.216421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211822 2562 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211825 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211828 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211832 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211835 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211837 2562 flags.go:64] FLAG: --lock-file="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211840 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211843 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211846 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211852 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211855 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211858 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211861 2562 flags.go:64] FLAG: --logging-format="text" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211864 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211867 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211869 2562 flags.go:64] FLAG: --manifest-url="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211872 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211877 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211880 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211884 2562 flags.go:64] FLAG: --max-pods="110" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211887 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211890 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211893 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211896 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211899 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:22:26.217029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211901 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211904 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211911 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211914 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211916 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211919 2562 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211922 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211927 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211930 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211933 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211936 2562 flags.go:64] FLAG: --port="10250" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211939 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211942 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee9201919bcd547b" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211946 2562 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211949 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211952 2562 flags.go:64] FLAG: --register-node="true" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211956 2562 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211960 2562 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211963 2562 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211966 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211968 2562 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211972 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211975 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211977 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211980 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:22:26.217633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211984 2562 flags.go:64] FLAG: --runonce="false" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211987 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211990 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211993 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211996 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.211999 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212013 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212017 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212020 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212023 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212026 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212028 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212031 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212034 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212037 2562 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212039 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212044 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212047 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212050 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212054 2562 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212057 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212059 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212062 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212067 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212070 2562 flags.go:64] FLAG: --v="2" Jul 02 23:22:26.218253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212075 2562 flags.go:64] FLAG: --version="false" Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212079 2562 flags.go:64] FLAG: --vmodule="" Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212083 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212086 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212208 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212213 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212216 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212219 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212222 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212225 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212228 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212231 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212235 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212239 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212242 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212244 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212247 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212249 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212252 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.218885 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212255 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212259 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212262 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212264 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212267 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212270 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212273 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212275 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212278 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212280 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212282 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212286 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212288 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212291 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212293 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212296 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212298 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212301 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212304 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212306 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.219364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212309 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212311 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212314 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212317 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212319 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212322 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212324 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212327 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212329 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212332 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212334 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212336 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212338 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212341 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212343 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212345 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212348 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212350 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212354 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212357 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.219854 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212359 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212362 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212364 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212367 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212370 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212372 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212374 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212377 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212379 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212382 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212384 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212387 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212389 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212391 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212394 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212397 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212400 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212402 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212405 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212407 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.220450 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212410 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212412 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212414 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212416 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212419 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212421 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212424 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212426 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212429 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212431 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212435 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212438 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212440 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212443 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212445 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212449 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212452 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212455 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212457 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212460 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.221264 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212462 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.221892 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212465 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.221892 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212467 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.221892 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.212470 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.221892 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.212475 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:26.222643 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.222623 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:22:26.222674 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.222644 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:22:26.222674 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.222666 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:22:26.222730 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.222675 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:22:26.222761 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222751 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.222761 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222755 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.222761 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222759 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.222761 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222762 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222764 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222768 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222771 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222775 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222777 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222780 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222784 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222787 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222790 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222792 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222795 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222798 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222800 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222803 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222805 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222807 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222818 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222822 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222825 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.222865 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222827 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222830 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222832 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222835 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222837 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222839 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222841 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222845 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222849 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222852 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222855 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222857 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222860 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222863 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222867 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222871 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222873 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222876 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222879 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.223370 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222881 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222884 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222886 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222889 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222891 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222894 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222896 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222900 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222902 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222904 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222907 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222910 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222913 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222916 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222918 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222921 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222923 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222925 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222928 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222930 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.223917 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222932 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222935 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222937 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222940 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222942 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222945 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222947 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222950 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222953 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222955 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222957 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222960 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222962 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222965 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222967 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222969 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222971 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222974 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222977 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222979 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.224494 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222982 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222984 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222986 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222989 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222991 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222994 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222996 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.222998 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223017 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223020 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223022 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223025 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223028 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223030 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223032 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223035 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.224980 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223039 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.223044 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223173 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223178 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223181 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223185 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223188 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223191 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223193 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223196 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223199 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223202 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223205 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223207 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223210 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:22:26.225376 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223213 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223216 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223219 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223221 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223223 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223226 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223228 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223231 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223233 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223236 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223238 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223240 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223243 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223247 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223250 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223253 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223255 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223259 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223262 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:22:26.225737 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223265 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223267 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223270 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223273 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223275 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223278 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223280 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223282 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223285 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223287 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223290 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223293 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223296 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223299 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223302 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223304 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223307 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223309 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223311 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:22:26.226204 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223314 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223316 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223318 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223320 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223323 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223325 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223328 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223330 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223332 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223334 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223337 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223340 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223343 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223345 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223347 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223350 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223352 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223354 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223357 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223359 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:22:26.226717 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223362 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223364 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223366 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223369 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223371 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223374 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223376 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223383 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223385 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223388 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223390 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223393 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223395 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223397 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223400 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223402 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223404 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223406 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223409 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223411 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223414 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:22:26.227242 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223416 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223419 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223421 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223423 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223426 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223428 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:26.223431 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.223435 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.223547 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:22:26.227742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.226322 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:22:26.227974 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.227903 2562 server.go:1019] "Starting client certificate rotation" Jul 02 23:22:26.228058 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.228038 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:22:26.228108 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.228081 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:22:26.230608 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.230580 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:22:26.231783 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.231763 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:22:26.243095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.243020 2562 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:22:26.245192 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.245174 2562 log.go:25] "Validated CRI v1 image API" Jul 02 23:22:26.246792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.246778 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:22:26.249336 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.249313 2562 fs.go:135] Filesystem UUIDs: map[1c0e5d7b-6057-446f-a2cd-4202a7dc679f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ad0dbd33-46d8-4225-b6b2-e82cf9fc550d:/dev/nvme0n1p3] Jul 02 23:22:26.249392 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.249337 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:22:26.256274 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.256065 2562 manager.go:216] Machine: {Timestamp:2026-07-02 23:22:26.254650087 +0000 UTC m=+0.339251496 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104696 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24b776b357f0204e7fb37a17c86bd0 SystemUUID:ec24b776-b357-f020-4e7f-b37a17c86bd0 BootID:a06fafa3-744b-4ebc-9a75-ab359cba9425 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:4b:a9:9b:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:4b:a9:9b:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:23:49:0c:c5:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:22:26.256274 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.256272 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:22:26.256385 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.256256 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:22:26.256385 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.256378 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:22:26.259960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.259938 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:22:26.260119 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.259963 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:22:26.260708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.260699 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:22:26.260738 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.260711 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:22:26.260738 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.260720 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:22:26.261348 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.261338 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:22:26.262748 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.262739 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:22:26.265546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.265535 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:22:26.265839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.265830 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:22:26.267280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.266272 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:22:26.267280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.266303 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:22:26.267280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.266326 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:22:26.267280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.266340 2562 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:22:26.267280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.266356 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:22:26.267555 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.267387 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:22:26.267555 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.267419 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:22:26.270252 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.270230 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:22:26.272124 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.272111 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:22:26.272183 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.272141 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:22:26.273290 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273279 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273296 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273302 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273308 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273316 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273323 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273334 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273339 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:22:26.273345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273346 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:22:26.273541 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273351 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:22:26.273541 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273359 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:22:26.273541 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.273364 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:22:26.274144 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.274133 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:22:26.274180 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.274145 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:22:26.277700 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.277680 2562 server.go:1268] "Started kubelet" Jul 02 23:22:26.277797 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.277765 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:22:26.277961 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.277914 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:22:26.278061 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.277986 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:22:26.278731 ip-10-0-130-160 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:22:26.279616 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.279584 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:22:26.279980 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.279961 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:22:26.280085 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.279973 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:22:26.280202 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.280183 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:22:26.283121 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.283104 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:22:26.283184 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.283122 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:22:26.285113 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.284040 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-160.ec2.internal.18be9c7911211a54 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-160.ec2.internal,UID:ip-10-0-130-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-160.ec2.internal,},FirstTimestamp:2026-07-02 23:22:26.277653076 +0000 UTC m=+0.362254485,LastTimestamp:2026-07-02 23:22:26.277653076 +0000 UTC m=+0.362254485,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-160.ec2.internal,}" Jul 02 23:22:26.289252 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.289235 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:22:26.289366 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.289346 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.289764 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.289326 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:22:26.289872 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.289855 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:22:26.290082 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290063 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:22:26.290082 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290081 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:22:26.290237 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290197 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:22:26.290237 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290207 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:22:26.290958 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290938 2562 factory.go:55] Registering systemd factory Jul 02 23:22:26.290958 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.290960 2562 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:22:26.291242 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291225 2562 factory.go:153] Registering CRI-O factory Jul 02 23:22:26.291316 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291247 2562 factory.go:223] Registration of the crio container factory successfully Jul 02 23:22:26.291316 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291297 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:22:26.291409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291321 2562 factory.go:103] Registering Raw factory Jul 02 23:22:26.291409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291335 2562 manager.go:1196] Started watching for new ooms in manager Jul 02 23:22:26.291792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.291779 2562 manager.go:349] Starting recovery of all containers Jul 02 23:22:26.293187 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.293155 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 23:22:26.293301 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.293279 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 23:22:26.302507 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.302359 2562 manager.go:354] Recovery completed Jul 02 23:22:26.303549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.303527 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djxvd" Jul 02 23:22:26.306880 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.306868 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.308893 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.308878 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.308947 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.308909 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.308947 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.308923 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.309429 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.309419 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:22:26.309429 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.309429 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:22:26.309501 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.309431 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.309501 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.309444 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:22:26.310115 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310102 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.310169 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310126 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.310169 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310135 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.310498 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310484 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djxvd" Jul 02 23:22:26.310630 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310619 2562 policy_none.go:49] "None policy: Start" Jul 02 23:22:26.310660 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310635 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:22:26.310660 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.310645 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:22:26.311340 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.311243 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-160.ec2.internal.18be9c7912fdcbfe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-160.ec2.internal,UID:ip-10-0-130-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-160.ec2.internal,},FirstTimestamp:2026-07-02 23:22:26.308893694 +0000 UTC m=+0.393495102,LastTimestamp:2026-07-02 23:22:26.308893694 +0000 UTC m=+0.393495102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-160.ec2.internal,}" Jul 02 23:22:26.312121 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.312106 2562 policy_none.go:47] "Start" Jul 02 23:22:26.351726 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.351698 2562 manager.go:337] "Starting Device Plugin manager" Jul 02 23:22:26.351842 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.351743 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:22:26.351842 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.351754 2562 server.go:87] "Starting device plugin registration server" Jul 02 23:22:26.352149 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.352136 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:22:26.352199 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.352152 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:22:26.352278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.352263 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:22:26.352374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.352363 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:22:26.352374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.352373 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:22:26.352942 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.352923 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:22:26.353054 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.352967 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.424546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.424504 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:22:26.425997 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.425971 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:22:26.426145 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.426033 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:22:26.426145 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.426067 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:22:26.426145 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.426110 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:22:26.429373 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.429354 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:26.452352 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.452278 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.453333 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.453315 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.453426 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.453349 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.453426 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.453367 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.453426 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.453395 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.462129 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.462101 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.462219 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.462132 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-160.ec2.internal\": node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.526644 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.526608 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal"] Jul 02 23:22:26.526793 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.526699 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.526793 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.526753 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.527719 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.527703 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.527776 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.527731 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.527776 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.527741 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.529118 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.529105 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.529229 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.529214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.529281 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.529241 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.530115 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530089 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.530115 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530112 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.530229 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530125 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.530229 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530130 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.530229 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530152 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.530229 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.530165 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.531519 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.531504 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.531559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.531538 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:22:26.532277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.532258 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:22:26.532334 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.532294 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:22:26.532334 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.532310 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:22:26.548648 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.548624 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-160.ec2.internal\" not found" node="ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.552934 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.552916 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-160.ec2.internal\" not found" node="ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.591222 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.591193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5e8112810e8b8f02dbe6afdacc66d20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-160.ec2.internal\" (UID: \"a5e8112810e8b8f02dbe6afdacc66d20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.591222 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.591222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.591379 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.591238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.627196 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.627162 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.692132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5e8112810e8b8f02dbe6afdacc66d20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-160.ec2.internal\" (UID: \"a5e8112810e8b8f02dbe6afdacc66d20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.692132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.692323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.692323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692196 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5e8112810e8b8f02dbe6afdacc66d20-config\") pod \"kube-apiserver-proxy-ip-10-0-130-160.ec2.internal\" (UID: \"a5e8112810e8b8f02dbe6afdacc66d20\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.692323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692196 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.692323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.692203 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/115aa49a038e47dac34d813a83b88e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal\" (UID: \"115aa49a038e47dac34d813a83b88e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.728280 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.728216 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.829100 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.829056 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:26.852247 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.852225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.856048 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:26.856030 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:26.929476 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:26.929441 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.030048 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:27.029965 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.130781 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:27.130734 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.227196 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.227166 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:22:27.227806 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.227328 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:22:27.231648 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:27.231630 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.312483 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.312364 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:17:26 +0000 UTC" deadline="2028-01-06 00:52:46.150602206 +0000 UTC" Jul 02 23:22:27.312483 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.312417 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13249h30m18.838189077s" Jul 02 23:22:27.333098 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:27.333061 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.434171 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:27.434141 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-160.ec2.internal\" not found" Jul 02 23:22:27.482805 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:27.482775 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e8112810e8b8f02dbe6afdacc66d20.slice/crio-a15a1968af7ee39a5679626fe48b1ba34d344a13b8bc6b539b6ffae7e06896ac WatchSource:0}: Error finding container a15a1968af7ee39a5679626fe48b1ba34d344a13b8bc6b539b6ffae7e06896ac: Status 404 returned error can't find the container with id a15a1968af7ee39a5679626fe48b1ba34d344a13b8bc6b539b6ffae7e06896ac Jul 02 23:22:27.483270 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:27.483243 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115aa49a038e47dac34d813a83b88e47.slice/crio-ff89cc766b0d7f4bc1e794f98d2bdf1293d8d823cc28bde7751e11d52a8f24fc WatchSource:0}: Error finding container ff89cc766b0d7f4bc1e794f98d2bdf1293d8d823cc28bde7751e11d52a8f24fc: Status 404 returned error can't find the container with id ff89cc766b0d7f4bc1e794f98d2bdf1293d8d823cc28bde7751e11d52a8f24fc Jul 02 23:22:27.487694 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.487676 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:22:27.508150 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.507520 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:27.592667 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.592569 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" Jul 02 23:22:27.609586 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.609562 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:22:27.611748 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.611733 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" Jul 02 23:22:27.620393 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.620372 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:22:27.681830 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.681793 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:27.838233 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:27.838199 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:28.268539 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.268506 2562 apiserver.go:52] "Watching apiserver" Jul 02 23:22:28.278246 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.278211 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:22:28.278833 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.278803 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-k94b5","openshift-multus/network-metrics-daemon-wq5cf","openshift-network-diagnostics/network-check-target-jc2vq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b","openshift-cluster-node-tuning-operator/tuned-tv9sw","openshift-image-registry/node-ca-r2fwr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal","openshift-network-operator/iptables-alerter-vht9p","openshift-ovn-kubernetes/ovnkube-node-bdq8z","kube-system/global-pull-secret-syncer-q5gzx","kube-system/konnectivity-agent-ghrxx","kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal","openshift-dns/node-resolver-hxd76","openshift-multus/multus-7jnk5"] Jul 02 23:22:28.280708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.280685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.282756 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.282209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.283112 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g47tw\"" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.283603 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.283783 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.284024 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.284682 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.285053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.284905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mjhb9\"" Jul 02 23:22:28.285415 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285270 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.285487 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285460 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:22:28.285770 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285664 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:22:28.285770 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285737 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.285905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:28.285905 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.285813 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:28.285905 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.285834 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:28.285905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.285774 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:22:28.287577 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.287552 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.289077 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.289002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.291127 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.290178 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4rgtx\"" Jul 02 23:22:28.291127 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.290422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.291127 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.290682 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.291127 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.290788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.291127 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.291087 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300176 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300268 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qksbw\"" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300315 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.300639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300176 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-dbus\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-registration-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-systemd\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300961 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.300996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-binary-copy\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.301034 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-kubernetes\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysconfig\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-var-lib-kubelet\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-tmp\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4155ac59-392e-4cc9-816a-17b09bb46339-host-slash\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-host\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvkcc\" (UniqueName: \"kubernetes.io/projected/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-kube-api-access-jvkcc\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.301407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-system-cni-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-cnibin\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-os-release\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-lib-modules\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-socket-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-device-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-host\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4155ac59-392e-4cc9-816a-17b09bb46339-iptables-alerter-script\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz29v\" (UniqueName: \"kubernetes.io/projected/4155ac59-392e-4cc9-816a-17b09bb46339-kube-api-access-bz29v\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.301817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-serviceca\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7s5\" (UniqueName: \"kubernetes.io/projected/96503c4a-6589-43be-8d4a-7904276ff7ee-kube-api-access-sj7s5\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-kubelet-config\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-conf\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.301979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-run\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-sys-fs\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9rqt\" (UniqueName: \"kubernetes.io/projected/ed8fda13-208f-47f1-bba0-e3f41fb34716-kube-api-access-j9rqt\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.302121 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-modprobe-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-sys\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-tuned\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2r2\" (UniqueName: \"kubernetes.io/projected/9bc03e29-0523-40d8-ada0-0dd7ff498c59-kube-api-access-9d2r2\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.302792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.302792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302427 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.302882 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302858 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.302882 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.302867 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ftggf\"" Jul 02 23:22:28.303875 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.303853 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:22:28.304080 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304065 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.304158 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.304546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gbb7s\"" Jul 02 23:22:28.304655 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304634 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:22:28.304794 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304777 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:22:28.304890 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304822 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:22:28.305006 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.304944 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:22:28.305360 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.305231 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.307445 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.306937 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:22:28.307445 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.307200 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:22:28.307445 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.307374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.307445 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.307396 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rbs4z\"" Jul 02 23:22:28.307751 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.307731 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.310242 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.310054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:22:28.310242 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.310119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mfnms\"" Jul 02 23:22:28.310409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.310395 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:22:28.311501 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.311480 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xdq4h\"" Jul 02 23:22:28.312007 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.311973 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:22:28.313803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.313781 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:22:28.336660 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.336632 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5w849" Jul 02 23:22:28.344593 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.344571 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5w849" Jul 02 23:22:28.391109 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.391013 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:22:28.405436 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cni-binary-copy\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.405590 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvkcc\" (UniqueName: \"kubernetes.io/projected/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-kube-api-access-jvkcc\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.405590 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-system-cni-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.405590 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-os-release\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.405590 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2gxt\" (UniqueName: \"kubernetes.io/projected/795d14f0-d610-4fb2-9ccc-adc4f3478766-kube-api-access-k2gxt\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.405590 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-device-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-var-lib-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-config\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-script-lib\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cnibin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-conf\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.405774 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405762 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b329e697-8cf2-492f-b466-344b099f79c5-ovn-node-metrics-cert\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-etc-kubernetes\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-system-cni-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405861 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9rqt\" (UniqueName: \"kubernetes.io/projected/ed8fda13-208f-47f1-bba0-e3f41fb34716-kube-api-access-j9rqt\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405886 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-modprobe-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-sys\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-tuned\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.405984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-conf\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406020 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2r2\" (UniqueName: \"kubernetes.io/projected/9bc03e29-0523-40d8-ada0-0dd7ff498c59-kube-api-access-9d2r2\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406107 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-device-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-socket-dir-parent\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.406349 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-dbus\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx69c\" (UniqueName: \"kubernetes.io/projected/69b0b64a-23c1-45fb-84ba-9abd7c29442f-kube-api-access-lx69c\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.406441 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:22:28.906407743 +0000 UTC m=+2.991009142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-log-socket\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjbq\" (UniqueName: \"kubernetes.io/projected/b329e697-8cf2-492f-b466-344b099f79c5-kube-api-access-fjjbq\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.406568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-binary-copy\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406671 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-modprobe-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406678 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b0b64a-23c1-45fb-84ba-9abd7c29442f-hosts-file\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-slash\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-node-log\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406753 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-os-release\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-env-overrides\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysconfig\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-var-lib-kubelet\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4155ac59-392e-4cc9-816a-17b09bb46339-host-slash\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-ovn\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-bin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.406983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-hostroot\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.407314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-conf-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-host\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-cnibin\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-lib-modules\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-kubelet\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407191 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-socket-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-host\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4155ac59-392e-4cc9-816a-17b09bb46339-iptables-alerter-script\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz29v\" (UniqueName: \"kubernetes.io/projected/4155ac59-392e-4cc9-816a-17b09bb46339-kube-api-access-bz29v\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407298 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-agent-certs\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-k8s-cni-cncf-io\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5nh\" (UniqueName: \"kubernetes.io/projected/0c371128-dafc-4f47-a55c-7ae0e7ceeced-kube-api-access-bk5nh\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-serviceca\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7s5\" (UniqueName: \"kubernetes.io/projected/96503c4a-6589-43be-8d4a-7904276ff7ee-kube-api-access-sj7s5\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-kubelet-config\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-run\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.408126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-netns\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-multus\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-sys\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-sys-fs\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-systemd-units\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-systemd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-os-release\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-multus-certs\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-bin\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-konnectivity-ca\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-registration-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-systemd\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407907 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-netns\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-etc-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.408798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.407997 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-system-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-daemon-config\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-kubernetes\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b0b64a-23c1-45fb-84ba-9abd7c29442f-tmp-dir\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-kubelet\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-tmp\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-netd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408444 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-registration-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408511 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-systemd\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408614 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-kubernetes\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408731 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.408885 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-dbus\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.409016 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysctl-d\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.409156 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.409778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.409688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8fda13-208f-47f1-bba0-e3f41fb34716-cni-binary-copy\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.409858 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-sys-fs\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410378 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8fda13-208f-47f1-bba0-e3f41fb34716-cnibin\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410478 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-sysconfig\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-var-lib-kubelet\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.410647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410576 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4155ac59-392e-4cc9-816a-17b09bb46339-host-slash\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.411011 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-host\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.411011 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410730 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.411011 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-lib-modules\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.411011 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.410984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96503c4a-6589-43be-8d4a-7904276ff7ee-socket-dir\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.411230 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.411052 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-host\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.412289 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.411609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4155ac59-392e-4cc9-816a-17b09bb46339-iptables-alerter-script\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.412289 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.412144 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9bc03e29-0523-40d8-ada0-0dd7ff498c59-run\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.412289 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.412229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-kubelet-config\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.412289 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.412274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-serviceca\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.412802 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.412775 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-tmp\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.413284 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.413260 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9bc03e29-0523-40d8-ada0-0dd7ff498c59-etc-tuned\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.415523 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.415425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvkcc\" (UniqueName: \"kubernetes.io/projected/cdd68c25-ded4-49cb-bf2a-d85ac74f44d3-kube-api-access-jvkcc\") pod \"node-ca-r2fwr\" (UID: \"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3\") " pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.418441 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.417252 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:28.418441 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.417275 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:28.418441 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.417291 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:28.418441 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.417394 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:22:28.917374116 +0000 UTC m=+3.001975518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:28.422549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.421946 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7s5\" (UniqueName: \"kubernetes.io/projected/96503c4a-6589-43be-8d4a-7904276ff7ee-kube-api-access-sj7s5\") pod \"aws-ebs-csi-driver-node-57c2b\" (UID: \"96503c4a-6589-43be-8d4a-7904276ff7ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.422549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.422324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz29v\" (UniqueName: \"kubernetes.io/projected/4155ac59-392e-4cc9-816a-17b09bb46339-kube-api-access-bz29v\") pod \"iptables-alerter-vht9p\" (UID: \"4155ac59-392e-4cc9-816a-17b09bb46339\") " pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.433985 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.433957 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9rqt\" (UniqueName: \"kubernetes.io/projected/ed8fda13-208f-47f1-bba0-e3f41fb34716-kube-api-access-j9rqt\") pod \"multus-additional-cni-plugins-k94b5\" (UID: \"ed8fda13-208f-47f1-bba0-e3f41fb34716\") " pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.437776 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.437744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2r2\" (UniqueName: \"kubernetes.io/projected/9bc03e29-0523-40d8-ada0-0dd7ff498c59-kube-api-access-9d2r2\") pod \"tuned-tv9sw\" (UID: \"9bc03e29-0523-40d8-ada0-0dd7ff498c59\") " pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.440790 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.440740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" event={"ID":"115aa49a038e47dac34d813a83b88e47","Type":"ContainerStarted","Data":"ff89cc766b0d7f4bc1e794f98d2bdf1293d8d823cc28bde7751e11d52a8f24fc"} Jul 02 23:22:28.442148 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.442120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" event={"ID":"a5e8112810e8b8f02dbe6afdacc66d20","Type":"ContainerStarted","Data":"a15a1968af7ee39a5679626fe48b1ba34d344a13b8bc6b539b6ffae7e06896ac"} Jul 02 23:22:28.508729 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2gxt\" (UniqueName: \"kubernetes.io/projected/795d14f0-d610-4fb2-9ccc-adc4f3478766-kube-api-access-k2gxt\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-var-lib-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-config\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-script-lib\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-var-lib-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cnibin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b329e697-8cf2-492f-b466-344b099f79c5-ovn-node-metrics-cert\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.508929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508922 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-etc-kubernetes\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.508871 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cnibin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509007 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-etc-kubernetes\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-socket-dir-parent\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx69c\" (UniqueName: \"kubernetes.io/projected/69b0b64a-23c1-45fb-84ba-9abd7c29442f-kube-api-access-lx69c\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-log-socket\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjbq\" (UniqueName: \"kubernetes.io/projected/b329e697-8cf2-492f-b466-344b099f79c5-kube-api-access-fjjbq\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b0b64a-23c1-45fb-84ba-9abd7c29442f-hosts-file\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-slash\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-node-log\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509333 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-env-overrides\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-ovn\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509384 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-bin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509439 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-bin\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509456 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-socket-dir-parent\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-slash\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-config\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-node-log\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-hostroot\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-hostroot\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b0b64a-23c1-45fb-84ba-9abd7c29442f-hosts-file\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-conf-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-kubelet\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509598 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-ovn\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-agent-certs\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509633 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-k8s-cni-cncf-io\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5nh\" (UniqueName: \"kubernetes.io/projected/0c371128-dafc-4f47-a55c-7ae0e7ceeced-kube-api-access-bk5nh\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-netns\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.509991 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.509729 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509762 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-multus\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.509797 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:29.009777606 +0000 UTC m=+3.094379007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-conf-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-cni-multus\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509839 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-var-lib-kubelet\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-systemd-units\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-systemd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-os-release\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-multus-certs\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509951 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-bin\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-konnectivity-ca\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-netns\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-etc-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510066 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-system-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-os-release\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.510846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-daemon-config\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b0b64a-23c1-45fb-84ba-9abd7c29442f-tmp-dir\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-kubelet\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510250 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-k8s-cni-cncf-io\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-netd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510290 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-netd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cni-binary-copy\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.509562 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-log-socket\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510390 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-systemd-units\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510468 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-run-systemd\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-netns\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-run-netns\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510564 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-host-run-multus-certs\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-cni-bin\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510656 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c371128-dafc-4f47-a55c-7ae0e7ceeced-system-cni-dir\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-host-kubelet\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b329e697-8cf2-492f-b466-344b099f79c5-etc-openvswitch\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.511708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510703 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b0b64a-23c1-45fb-84ba-9abd7c29442f-tmp-dir\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.510940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-cni-binary-copy\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.511160 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-env-overrides\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.511248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c371128-dafc-4f47-a55c-7ae0e7ceeced-multus-daemon-config\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.511284 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-konnectivity-ca\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.511285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b329e697-8cf2-492f-b466-344b099f79c5-ovnkube-script-lib\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.512551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.511950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b329e697-8cf2-492f-b466-344b099f79c5-ovn-node-metrics-cert\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.513157 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.513139 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4ada550-0bb9-4c82-bf50-e9cf09dd0713-agent-certs\") pod \"konnectivity-agent-ghrxx\" (UID: \"c4ada550-0bb9-4c82-bf50-e9cf09dd0713\") " pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.521818 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.521744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2gxt\" (UniqueName: \"kubernetes.io/projected/795d14f0-d610-4fb2-9ccc-adc4f3478766-kube-api-access-k2gxt\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:28.522456 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.522428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjbq\" (UniqueName: \"kubernetes.io/projected/b329e697-8cf2-492f-b466-344b099f79c5-kube-api-access-fjjbq\") pod \"ovnkube-node-bdq8z\" (UID: \"b329e697-8cf2-492f-b466-344b099f79c5\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.522599 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.522482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx69c\" (UniqueName: \"kubernetes.io/projected/69b0b64a-23c1-45fb-84ba-9abd7c29442f-kube-api-access-lx69c\") pod \"node-resolver-hxd76\" (UID: \"69b0b64a-23c1-45fb-84ba-9abd7c29442f\") " pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.523084 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.523057 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5nh\" (UniqueName: \"kubernetes.io/projected/0c371128-dafc-4f47-a55c-7ae0e7ceeced-kube-api-access-bk5nh\") pod \"multus-7jnk5\" (UID: \"0c371128-dafc-4f47-a55c-7ae0e7ceeced\") " pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.537685 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.537661 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:22:28.601773 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.601741 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" Jul 02 23:22:28.613591 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.613557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k94b5" Jul 02 23:22:28.622919 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.622891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" Jul 02 23:22:28.630176 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.630153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r2fwr" Jul 02 23:22:28.639117 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.639093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vht9p" Jul 02 23:22:28.648454 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.648432 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:28.657404 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.657368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:28.664373 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.664351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hxd76" Jul 02 23:22:28.677278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.677246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7jnk5" Jul 02 23:22:28.914585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:28.914506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:28.914757 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.914657 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:28.914757 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:28.914735 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:22:29.914712718 +0000 UTC m=+3.999314117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:29.015769 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.015724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:29.015944 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.015826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:29.016008 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.015943 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:29.016008 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.015970 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:29.016008 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.015985 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:29.016171 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.015944 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:29.016171 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.016059 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:22:30.016040038 +0000 UTC m=+4.100641452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:29.016171 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.016083 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:30.016069301 +0000 UTC m=+4.100670699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:29.347367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.347323 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:17:28 +0000 UTC" deadline="2028-04-10 01:04:00.523746578 +0000 UTC" Jul 02 23:22:29.347367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.347359 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15529h41m31.176391116s" Jul 02 23:22:29.426607 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.426570 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:29.426786 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.426692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:29.426846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.426770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:29.426910 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.426889 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:29.663892 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.663858 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb329e697_8cf2_492f_b466_344b099f79c5.slice/crio-90bc09e73d24b6b874c9f4d22773334290fba7eb4627e4ea4d37a49c33d7c0fb WatchSource:0}: Error finding container 90bc09e73d24b6b874c9f4d22773334290fba7eb4627e4ea4d37a49c33d7c0fb: Status 404 returned error can't find the container with id 90bc09e73d24b6b874c9f4d22773334290fba7eb4627e4ea4d37a49c33d7c0fb Jul 02 23:22:29.665514 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.665451 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc03e29_0523_40d8_ada0_0dd7ff498c59.slice/crio-12c74636f5af3f436134df2cdd5aa37251d43f797b62e04c891397549cfaf45e WatchSource:0}: Error finding container 12c74636f5af3f436134df2cdd5aa37251d43f797b62e04c891397549cfaf45e: Status 404 returned error can't find the container with id 12c74636f5af3f436134df2cdd5aa37251d43f797b62e04c891397549cfaf45e Jul 02 23:22:29.670176 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.669925 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c371128_dafc_4f47_a55c_7ae0e7ceeced.slice/crio-7da51738ab0d31dfb8df3103f0443de6dca48082e8d8392fe91ac5bba24607c6 WatchSource:0}: Error finding container 7da51738ab0d31dfb8df3103f0443de6dca48082e8d8392fe91ac5bba24607c6: Status 404 returned error can't find the container with id 7da51738ab0d31dfb8df3103f0443de6dca48082e8d8392fe91ac5bba24607c6 Jul 02 23:22:29.670984 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.670961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b0b64a_23c1_45fb_84ba_9abd7c29442f.slice/crio-a030bbcb428ffd9e3d5ecd082412091249ab04d6ee4cf2eceeb6e3b139f23630 WatchSource:0}: Error finding container a030bbcb428ffd9e3d5ecd082412091249ab04d6ee4cf2eceeb6e3b139f23630: Status 404 returned error can't find the container with id a030bbcb428ffd9e3d5ecd082412091249ab04d6ee4cf2eceeb6e3b139f23630 Jul 02 23:22:29.672253 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.672213 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96503c4a_6589_43be_8d4a_7904276ff7ee.slice/crio-49868472a3c075182ee70b37caae5b48f9e82604dcbcbb43a0a470256dd315cb WatchSource:0}: Error finding container 49868472a3c075182ee70b37caae5b48f9e82604dcbcbb43a0a470256dd315cb: Status 404 returned error can't find the container with id 49868472a3c075182ee70b37caae5b48f9e82604dcbcbb43a0a470256dd315cb Jul 02 23:22:29.672847 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.672758 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8fda13_208f_47f1_bba0_e3f41fb34716.slice/crio-ff42027399b8f11f1f9841b2ef48585dd93756730e8b65f4fe81683a90defea8 WatchSource:0}: Error finding container ff42027399b8f11f1f9841b2ef48585dd93756730e8b65f4fe81683a90defea8: Status 404 returned error can't find the container with id ff42027399b8f11f1f9841b2ef48585dd93756730e8b65f4fe81683a90defea8 Jul 02 23:22:29.677777 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.677755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ada550_0bb9_4c82_bf50_e9cf09dd0713.slice/crio-04f7a4e9927325967eded24237704bd11bf652e31c9a63b5b12baa779be166b4 WatchSource:0}: Error finding container 04f7a4e9927325967eded24237704bd11bf652e31c9a63b5b12baa779be166b4: Status 404 returned error can't find the container with id 04f7a4e9927325967eded24237704bd11bf652e31c9a63b5b12baa779be166b4 Jul 02 23:22:29.678084 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:22:29.677960 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4155ac59_392e_4cc9_816a_17b09bb46339.slice/crio-e8b4f6e23dc67ac06a222eb7ac9aec202e5092cae7bc8c8c1794fc48c4a8ab84 WatchSource:0}: Error finding container e8b4f6e23dc67ac06a222eb7ac9aec202e5092cae7bc8c8c1794fc48c4a8ab84: Status 404 returned error can't find the container with id e8b4f6e23dc67ac06a222eb7ac9aec202e5092cae7bc8c8c1794fc48c4a8ab84 Jul 02 23:22:29.924116 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:29.924086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:29.924224 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.924197 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:29.924282 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:29.924270 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:22:31.924251008 +0000 UTC m=+6.008852407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:30.025101 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.025056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:30.025288 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.025145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:30.025288 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025197 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:30.025288 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025269 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:30.025288 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025276 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:32.025257921 +0000 UTC m=+6.109859323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:30.025288 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025285 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:30.025502 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025297 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:30.025502 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.025337 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:22:32.02532508 +0000 UTC m=+6.109926477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:30.348148 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.347970 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:17:28 +0000 UTC" deadline="2028-01-15 21:38:26.93412266 +0000 UTC" Jul 02 23:22:30.348148 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.348054 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13486h15m56.586074067s" Jul 02 23:22:30.430493 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.429661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:30.430493 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:30.429812 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:30.451742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.451601 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hxd76" event={"ID":"69b0b64a-23c1-45fb-84ba-9abd7c29442f","Type":"ContainerStarted","Data":"a030bbcb428ffd9e3d5ecd082412091249ab04d6ee4cf2eceeb6e3b139f23630"} Jul 02 23:22:30.465945 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.465905 2562 generic.go:356] "Generic (PLEG): container finished" podID="115aa49a038e47dac34d813a83b88e47" containerID="d9cdf5fc2f657b4a0a962e6675a14ff2cabdd922dc5076afce6e8b52392205ae" exitCode=0 Jul 02 23:22:30.466164 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.466027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" event={"ID":"115aa49a038e47dac34d813a83b88e47","Type":"ContainerDied","Data":"d9cdf5fc2f657b4a0a962e6675a14ff2cabdd922dc5076afce6e8b52392205ae"} Jul 02 23:22:30.470452 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.470373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" event={"ID":"a5e8112810e8b8f02dbe6afdacc66d20","Type":"ContainerStarted","Data":"fd4753a21ec63cd2cb9162b62f0ad030f7aed4676a870da5b28239a7942a4918"} Jul 02 23:22:30.476259 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.476163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r2fwr" event={"ID":"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3","Type":"ContainerStarted","Data":"177497cbc63d668210303bf691d8703396562c5146fbdb321ad2ed5b8d315423"} Jul 02 23:22:30.489558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.489526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghrxx" event={"ID":"c4ada550-0bb9-4c82-bf50-e9cf09dd0713","Type":"ContainerStarted","Data":"04f7a4e9927325967eded24237704bd11bf652e31c9a63b5b12baa779be166b4"} Jul 02 23:22:30.502250 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.502217 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerStarted","Data":"ff42027399b8f11f1f9841b2ef48585dd93756730e8b65f4fe81683a90defea8"} Jul 02 23:22:30.536839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.536755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7jnk5" event={"ID":"0c371128-dafc-4f47-a55c-7ae0e7ceeced","Type":"ContainerStarted","Data":"7da51738ab0d31dfb8df3103f0443de6dca48082e8d8392fe91ac5bba24607c6"} Jul 02 23:22:30.550759 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.550432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" event={"ID":"9bc03e29-0523-40d8-ada0-0dd7ff498c59","Type":"ContainerStarted","Data":"12c74636f5af3f436134df2cdd5aa37251d43f797b62e04c891397549cfaf45e"} Jul 02 23:22:30.565099 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.564997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"90bc09e73d24b6b874c9f4d22773334290fba7eb4627e4ea4d37a49c33d7c0fb"} Jul 02 23:22:30.573244 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.573172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vht9p" event={"ID":"4155ac59-392e-4cc9-816a-17b09bb46339","Type":"ContainerStarted","Data":"e8b4f6e23dc67ac06a222eb7ac9aec202e5092cae7bc8c8c1794fc48c4a8ab84"} Jul 02 23:22:30.576462 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:30.576350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" event={"ID":"96503c4a-6589-43be-8d4a-7904276ff7ee","Type":"ContainerStarted","Data":"49868472a3c075182ee70b37caae5b48f9e82604dcbcbb43a0a470256dd315cb"} Jul 02 23:22:31.426343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:31.426297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:31.426837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:31.426345 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:31.426837 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:31.426453 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:31.426837 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:31.426584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:31.604571 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:31.604530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" event={"ID":"115aa49a038e47dac34d813a83b88e47","Type":"ContainerStarted","Data":"9c116e1be15ed4e1c16e62beb0bd1b06cb4b4f42471ff990bc0871858a8adc96"} Jul 02 23:22:31.627595 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:31.627508 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-160.ec2.internal" podStartSLOduration=3.627485155 podStartE2EDuration="3.627485155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:30.497236935 +0000 UTC m=+4.581838357" watchObservedRunningTime="2026-07-02 23:22:31.627485155 +0000 UTC m=+5.712086578" Jul 02 23:22:31.944237 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:31.943572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:31.944237 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:31.943756 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:31.944237 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:31.943827 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:22:35.943804591 +0000 UTC m=+10.028406013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:32.044136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:32.044205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044369 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044389 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044405 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044467 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:22:36.044448914 +0000 UTC m=+10.129050315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044865 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:32.044961 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.044915 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:36.044898679 +0000 UTC m=+10.129500083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:32.429053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:32.428964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:32.429542 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:32.429137 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:33.427123 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:33.427085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:33.427321 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:33.427212 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:33.427652 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:33.427629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:33.427764 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:33.427739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:34.428692 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:34.428191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:34.428692 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:34.428342 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:35.426492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:35.426447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:35.426492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:35.426466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:35.426713 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:35.426583 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:35.426766 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:35.426714 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:35.975806 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:35.975762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:35.976300 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:35.975933 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:35.976300 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:35.975995 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:22:43.975976692 +0000 UTC m=+18.060578087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:36.076486 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:36.076418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:36.076649 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:36.076529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:36.076708 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076660 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:36.076758 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076723 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:22:44.076705029 +0000 UTC m=+18.161306425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:36.076838 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076787 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:36.076838 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076797 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:36.076838 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076806 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:36.076838 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.076830 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:22:44.076821786 +0000 UTC m=+18.161423182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:36.427040 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:36.426950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:36.427191 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:36.427105 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:37.427705 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:37.427207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:37.427705 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:37.427242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:37.427705 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:37.427351 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:37.427705 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:37.427497 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:38.430470 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:38.429820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:38.430470 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:38.429990 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:39.427061 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:39.427028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:39.427224 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:39.427159 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:39.427224 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:39.427191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:39.427364 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:39.427279 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:40.426678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:40.426647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:40.427101 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:40.426772 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:41.427013 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:41.426965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:41.427013 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:41.427018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:41.427531 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:41.427100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:41.427531 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:41.427221 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:42.427265 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:42.427230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:42.427698 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:42.427366 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:43.427097 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:43.427065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:43.427352 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:43.427066 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:43.427352 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:43.427173 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:43.427352 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:43.427265 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:44.035522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:44.035469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:44.035765 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.035628 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:44.035765 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.035709 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:23:00.035688801 +0000 UTC m=+34.120290198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:22:44.136018 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:44.135966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:44.136197 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:44.136057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:44.136197 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136158 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:44.136320 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136237 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:00.136218256 +0000 UTC m=+34.220819861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:22:44.136320 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136235 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:22:44.136320 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136266 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:22:44.136320 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136281 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:44.136523 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.136340 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:23:00.136324053 +0000 UTC m=+34.220925448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:22:44.427417 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:44.427335 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:44.427841 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:44.427476 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:45.427176 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:45.427140 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:45.427402 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:45.427266 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:45.427402 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:45.427320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:45.427780 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:45.427418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:46.427354 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:46.427321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:46.427551 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:46.427426 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:47.426510 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:47.426477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:47.426701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:47.426567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:47.426701 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:47.426589 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:47.426701 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:47.426668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:48.427084 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:48.427048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:48.427618 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:48.427192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:49.427155 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.426828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:49.427813 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.426883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:49.427813 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:49.427222 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:49.427813 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:49.427333 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:49.669902 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.669864 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r2fwr" event={"ID":"cdd68c25-ded4-49cb-bf2a-d85ac74f44d3","Type":"ContainerStarted","Data":"fcda603a9d1fe223c04bff19560129f156325a562850166cd57c0e0b75dab9c5"} Jul 02 23:22:49.673927 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.673888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghrxx" event={"ID":"c4ada550-0bb9-4c82-bf50-e9cf09dd0713","Type":"ContainerStarted","Data":"88c1de4bb112f671c061a36ded3a545ebd65e020b82221ecf4d2ab2c9f219186"} Jul 02 23:22:49.675335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.675310 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="763e9c851190a8f9de3872f40c86b8f89ce02397af1d56b2ce09e06904ba1796" exitCode=0 Jul 02 23:22:49.675441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.675387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"763e9c851190a8f9de3872f40c86b8f89ce02397af1d56b2ce09e06904ba1796"} Jul 02 23:22:49.676749 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.676733 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7jnk5" event={"ID":"0c371128-dafc-4f47-a55c-7ae0e7ceeced","Type":"ContainerStarted","Data":"aea2d4e99af55dd69e1e814fcc5a3a3116c3f5d987c823f4c56b90008d32fff7"} Jul 02 23:22:49.678198 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.678176 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" event={"ID":"9bc03e29-0523-40d8-ada0-0dd7ff498c59","Type":"ContainerStarted","Data":"1b1abd0bac68cbc498f5b03dbad08e69a41ea44fbd5bd557c35b6f05a93d475b"} Jul 02 23:22:49.681049 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"5cec98760ac502933291f47e2688c70c09c8804f9c73119c2fd3396145ae78a9"} Jul 02 23:22:49.681132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"9751cf1693043270430780a2a7d242c08bdf4fcea246502ee7f98dfe67cad1c6"} Jul 02 23:22:49.681132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"24ecdc9eda4978a0fdf04f00ef67cc3aa0cc78433d05def02f8d9d74251c90c8"} Jul 02 23:22:49.681208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"49960bc473fb25f48132cee8ad127c14b30e5cafb81f5447de241d20c5bf48cb"} Jul 02 23:22:49.681208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"5bcae60d10c4e1e112ff0b8f2c6582a5f69ffc9f5f1afd6f1cd2dfdccd520c77"} Jul 02 23:22:49.681289 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.681233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"087a3d0bcbe08c8429e85887d3f8376b5d1025bc96546c25f2348ee9180d0824"} Jul 02 23:22:49.682722 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.682701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" event={"ID":"96503c4a-6589-43be-8d4a-7904276ff7ee","Type":"ContainerStarted","Data":"c3322048f705d52d822b6f80b09ee717d52a728ef6c6b3328b431ac3f6908f63"} Jul 02 23:22:49.683881 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.683861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hxd76" event={"ID":"69b0b64a-23c1-45fb-84ba-9abd7c29442f","Type":"ContainerStarted","Data":"b50f16236c054d294e0ab7b77f6f8ddd51cd406d5d5e355184e3cdcc0d276569"} Jul 02 23:22:49.684143 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.684112 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r2fwr" podStartSLOduration=3.737173402 podStartE2EDuration="22.684101455s" totalImagesPullingTime="18.946928053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:49.683918983 +0000 UTC m=+23.768520403" watchObservedRunningTime="2026-07-02 23:22:49.684101455 +0000 UTC m=+23.768702873" Jul 02 23:22:49.684202 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.684175 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-160.ec2.internal" podStartSLOduration=21.684169188 podStartE2EDuration="21.684169188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:31.627044782 +0000 UTC m=+5.711646205" watchObservedRunningTime="2026-07-02 23:22:49.684169188 +0000 UTC m=+23.768770606" Jul 02 23:22:49.698905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.698855 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ghrxx" podStartSLOduration=3.751617264 podStartE2EDuration="22.698842425s" totalImagesPullingTime="18.947225161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:49.698435782 +0000 UTC m=+23.783037200" watchObservedRunningTime="2026-07-02 23:22:49.698842425 +0000 UTC m=+23.783443843" Jul 02 23:22:49.733422 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.733372 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tv9sw" podStartSLOduration=3.773769635 podStartE2EDuration="22.733356161s" totalImagesPullingTime="18.959586526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:49.732958973 +0000 UTC m=+23.817560390" watchObservedRunningTime="2026-07-02 23:22:49.733356161 +0000 UTC m=+23.817957579" Jul 02 23:22:49.768972 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.768921 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hxd76" podStartSLOduration=3.814052264 podStartE2EDuration="22.768899948s" totalImagesPullingTime="18.954847684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:49.751543567 +0000 UTC m=+23.836144986" watchObservedRunningTime="2026-07-02 23:22:49.768899948 +0000 UTC m=+23.853501360" Jul 02 23:22:49.769187 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:49.769030 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7jnk5" podStartSLOduration=3.762119689 podStartE2EDuration="22.769020441s" totalImagesPullingTime="19.006900752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:49.768471106 +0000 UTC m=+23.853072548" watchObservedRunningTime="2026-07-02 23:22:49.769020441 +0000 UTC m=+23.853621850" Jul 02 23:22:50.341497 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.341475 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:22:50.370383 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.370259 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:22:50.341493022Z","UUID":"715f1963-4240-4e4c-af13-3afd5b821de7","Handler":null,"Name":"","Endpoint":""} Jul 02 23:22:50.371841 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.371824 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:22:50.371950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.371865 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:22:50.427165 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.427134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:50.427527 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:50.427252 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:50.687950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.687875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vht9p" event={"ID":"4155ac59-392e-4cc9-816a-17b09bb46339","Type":"ContainerStarted","Data":"a88cc21855bee1d2cb659864558c3c827f52bdf212e1838d057dfd41decd55d4"} Jul 02 23:22:50.689950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.689920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" event={"ID":"96503c4a-6589-43be-8d4a-7904276ff7ee","Type":"ContainerStarted","Data":"f1d2e6af713154b3c8056d5cffc33502ab138e804590aff54ff96b1f6a7b7cdf"} Jul 02 23:22:50.702506 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:50.702456 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vht9p" podStartSLOduration=4.747195121 podStartE2EDuration="23.702439819s" totalImagesPullingTime="18.955244698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:50.702062635 +0000 UTC m=+24.786664052" watchObservedRunningTime="2026-07-02 23:22:50.702439819 +0000 UTC m=+24.787041238" Jul 02 23:22:51.426880 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:51.426851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:51.427033 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:51.426859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:51.427033 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:51.426938 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:51.427108 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:51.427050 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:51.697519 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:51.697286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"37f9a823fc47f4a29af03898e14ec4d7ec7b0ff552ffb5b8ab9cff7a67c7cc50"} Jul 02 23:22:52.426513 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:52.426480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:52.426873 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:52.426630 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:52.701172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:52.701097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" event={"ID":"96503c4a-6589-43be-8d4a-7904276ff7ee","Type":"ContainerStarted","Data":"3a63f3587dfb007c165a55cc81295d742bc22167481ffd147f300652f3f23468"} Jul 02 23:22:52.716808 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:52.716765 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-57c2b" podStartSLOduration=5.146265418 podStartE2EDuration="26.716744528s" totalImagesPullingTime="21.57047911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:52.716325982 +0000 UTC m=+26.800927424" watchObservedRunningTime="2026-07-02 23:22:52.716744528 +0000 UTC m=+26.801345949" Jul 02 23:22:53.427282 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:53.427248 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:53.427282 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:53.427255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:53.427524 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:53.427381 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:53.427801 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:53.427777 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:54.357980 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:54.357944 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:54.358664 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:54.358648 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:54.427158 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:54.427116 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:54.427322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:54.427259 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:54.705387 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:54.705358 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:54.705852 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:54.705835 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ghrxx" Jul 02 23:22:55.426650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.426428 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:55.427191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.426485 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:55.427191 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:55.426670 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:55.427191 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:55.426769 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:55.709390 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.709307 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="f56af6eb2fecc3b8b841b60c7d25e127154d9c59f8f58f9124750878aac47838" exitCode=0 Jul 02 23:22:55.709390 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.709364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"f56af6eb2fecc3b8b841b60c7d25e127154d9c59f8f58f9124750878aac47838"} Jul 02 23:22:55.713252 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.713222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" event={"ID":"b329e697-8cf2-492f-b466-344b099f79c5","Type":"ContainerStarted","Data":"41b2d7cf9cc34b82fc0d1fc6e04f33d84e8b2e28e3b1ce209558734fdd7f39e4"} Jul 02 23:22:55.749845 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:55.749795 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" podStartSLOduration=9.714425164 podStartE2EDuration="28.749779771s" totalImagesPullingTime="19.035354607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:55.747764345 +0000 UTC m=+29.832365763" watchObservedRunningTime="2026-07-02 23:22:55.749779771 +0000 UTC m=+29.834381189" Jul 02 23:22:56.427979 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.427951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:56.428635 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:56.428072 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:56.715935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.715914 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 23:22:56.716559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.716539 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:56.716684 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.716603 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:56.735998 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.735832 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:56.736150 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.736130 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:56.885249 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.885215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q5gzx"] Jul 02 23:22:56.885392 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.885329 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:56.885437 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:56.885417 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:56.887840 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.887815 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jc2vq"] Jul 02 23:22:56.887981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.887931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:56.888104 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:56.888081 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:56.888440 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.888416 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wq5cf"] Jul 02 23:22:56.888599 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:56.888515 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:56.888667 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:56.888618 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:57.344536 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:57.344501 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:22:57.719922 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:57.719892 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="122b0ee32764b7b9d22e9cdc104936c980b9f83d2a98804e2ced3af9df8fedaf" exitCode=0 Jul 02 23:22:57.720301 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:57.719970 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"122b0ee32764b7b9d22e9cdc104936c980b9f83d2a98804e2ced3af9df8fedaf"} Jul 02 23:22:58.426769 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:58.426698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:22:58.426916 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:58.426698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:22:58.426916 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:58.426698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:22:58.426916 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:58.426875 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:22:58.427066 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:58.426825 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:22:58.427066 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:22:58.426922 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:22:59.728743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:59.728711 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="56b07a21d5e4b99101d86cc59c6c0553dda531cd265c6ad5d25ac8d3242a6269" exitCode=0 Jul 02 23:22:59.729382 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:22:59.728760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"56b07a21d5e4b99101d86cc59c6c0553dda531cd265c6ad5d25ac8d3242a6269"} Jul 02 23:23:00.051762 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.051670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:00.051892 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.051800 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:23:00.051892 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.051855 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret podName:694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba nodeName:}" failed. No retries permitted until 2026-07-02 23:23:32.051841151 +0000 UTC m=+66.136442546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret") pod "global-pull-secret-syncer-q5gzx" (UID: "694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:23:00.152593 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.152553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:00.152775 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.152609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:00.152775 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152709 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:23:00.152775 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152755 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:23:00.152775 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152770 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:23:00.152939 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152783 2562 projected.go:196] Error preparing data for projected volume kube-api-access-8wtm4 for pod openshift-network-diagnostics/network-check-target-jc2vq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:23:00.152939 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152785 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs podName:795d14f0-d610-4fb2-9ccc-adc4f3478766 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:32.152766751 +0000 UTC m=+66.237368170 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs") pod "network-metrics-daemon-wq5cf" (UID: "795d14f0-d610-4fb2-9ccc-adc4f3478766") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:23:00.152939 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.152831 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4 podName:0d35f306-64f6-47cb-bbfb-43ea24d281ff nodeName:}" failed. No retries permitted until 2026-07-02 23:23:32.152815797 +0000 UTC m=+66.237417215 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8wtm4" (UniqueName: "kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4") pod "network-check-target-jc2vq" (UID: "0d35f306-64f6-47cb-bbfb-43ea24d281ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:23:00.427269 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.427192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:00.427269 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.427215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:00.427466 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:00.427193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:00.427466 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.427302 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:23:00.427466 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.427381 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:23:00.427466 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:00.427448 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:23:02.426853 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.426755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:02.427376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.426762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:02.427376 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.426904 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jc2vq" podUID="0d35f306-64f6-47cb-bbfb-43ea24d281ff" Jul 02 23:23:02.427376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.426762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:02.427376 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.427154 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wq5cf" podUID="795d14f0-d610-4fb2-9ccc-adc4f3478766" Jul 02 23:23:02.427376 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.427334 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q5gzx" podUID="694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba" Jul 02 23:23:02.704587 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.703221 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-160.ec2.internal" event="NodeReady" Jul 02 23:23:02.704587 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.703361 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:23:02.734396 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.734369 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74cd958458-ccx49"] Jul 02 23:23:02.736668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.736647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.740627 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.740607 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 23:23:02.740727 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.740655 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j45fm\"" Jul 02 23:23:02.740727 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.740708 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 23:23:02.740944 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.740929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 23:23:02.757965 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.757740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 23:23:02.758490 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.758470 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2ckbs"] Jul 02 23:23:02.760367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.760347 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wtxgv"] Jul 02 23:23:02.760502 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.760477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.762070 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.762054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.763680 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.763428 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:23:02.763680 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.763443 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:23:02.763680 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.763476 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qpvbp\"" Jul 02 23:23:02.764659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.764640 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:23:02.764659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.764656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:23:02.765182 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.765160 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-24d9j\"" Jul 02 23:23:02.765421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.765401 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:23:02.771717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.771693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74cd958458-ccx49"] Jul 02 23:23:02.783297 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.783271 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ckbs"] Jul 02 23:23:02.787208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.787171 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtxgv"] Jul 02 23:23:02.873523 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a657f196-64df-4605-9d85-e726752bd25a-tmp-dir\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.873711 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn2s2\" (UniqueName: \"kubernetes.io/projected/a657f196-64df-4605-9d85-e726752bd25a-kube-api-access-rn2s2\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.873711 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.873711 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.873711 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.873923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-cert\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.873923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86sn9\" (UniqueName: \"kubernetes.io/projected/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-kube-api-access-86sn9\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.873923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.873923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppdb\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.873923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a657f196-64df-4605-9d85-e726752bd25a-config-volume\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.874178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.874178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.874178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.873999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.874178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.874043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.974934 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.974843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.974934 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.974891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.974934 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.974921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.974949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-cert\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.975238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.974994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86sn9\" (UniqueName: \"kubernetes.io/projected/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-kube-api-access-86sn9\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.975238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975238 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.975060 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:02.975238 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.975138 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:03.475114381 +0000 UTC m=+37.559715778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:02.975443 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppdb\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975443 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a657f196-64df-4605-9d85-e726752bd25a-config-volume\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.975443 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975443 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975443 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975394 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a657f196-64df-4605-9d85-e726752bd25a-tmp-dir\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn2s2\" (UniqueName: \"kubernetes.io/projected/a657f196-64df-4605-9d85-e726752bd25a-kube-api-access-rn2s2\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.975636 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:02.975657 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.975651 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:02.975915 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:02.975697 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:03.47568276 +0000 UTC m=+37.560284156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:02.975915 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.975867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a657f196-64df-4605-9d85-e726752bd25a-config-volume\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.976409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.976066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.976409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.976099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a657f196-64df-4605-9d85-e726752bd25a-tmp-dir\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.976539 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.976495 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.979954 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.979926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.979954 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.979948 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-cert\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.980107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.979952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.987093 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.987047 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86sn9\" (UniqueName: \"kubernetes.io/projected/cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f-kube-api-access-86sn9\") pod \"ingress-canary-wtxgv\" (UID: \"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f\") " pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:02.988154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.987336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn2s2\" (UniqueName: \"kubernetes.io/projected/a657f196-64df-4605-9d85-e726752bd25a-kube-api-access-rn2s2\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:02.988154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.987495 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:02.988154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:02.987820 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppdb\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:03.078627 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:03.078593 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtxgv" Jul 02 23:23:03.480597 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:03.480556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:03.480640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:03.480693 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:03.480760 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:03.480772 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:03.480780 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:04.480755944 +0000 UTC m=+38.565357355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:03.481059 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:03.480810 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:04.480796563 +0000 UTC m=+38.565397960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:04.288112 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.287861 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtxgv"] Jul 02 23:23:04.291583 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:04.291558 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2d2948_d60c_4fb4_8fed_aa89cf5b3d8f.slice/crio-a18753938f2d5ffb7990658e7b05036988071b55746fd819c3a669511670e75c WatchSource:0}: Error finding container a18753938f2d5ffb7990658e7b05036988071b55746fd819c3a669511670e75c: Status 404 returned error can't find the container with id a18753938f2d5ffb7990658e7b05036988071b55746fd819c3a669511670e75c Jul 02 23:23:04.427290 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.427192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:04.427441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.427192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:04.427441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.427191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:04.430021 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.429986 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:23:04.431145 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.430865 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z4l4l\"" Jul 02 23:23:04.431145 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.430884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p57lg\"" Jul 02 23:23:04.431312 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.431268 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:23:04.431364 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.431317 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:23:04.431524 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.431505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:23:04.488820 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.488786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.488847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:04.488942 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:04.488949 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:04.489038 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:06.489018731 +0000 UTC m=+40.573620144 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:04.488954 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:04.489322 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:04.489086 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:06.489074882 +0000 UTC m=+40.573676278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:04.746450 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.746370 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="267be763360ddedff9ddc1efbdd0933b79374f84ee4ef18c40a5f3ce12461bbd" exitCode=0 Jul 02 23:23:04.746771 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.746453 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"267be763360ddedff9ddc1efbdd0933b79374f84ee4ef18c40a5f3ce12461bbd"} Jul 02 23:23:04.752383 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:04.752351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtxgv" event={"ID":"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f","Type":"ContainerStarted","Data":"a18753938f2d5ffb7990658e7b05036988071b55746fd819c3a669511670e75c"} Jul 02 23:23:05.243417 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.243175 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s"] Jul 02 23:23:05.267093 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.267069 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s"] Jul 02 23:23:05.267244 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.267206 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.269700 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.269663 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 23:23:05.269700 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.269686 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 23:23:05.269872 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.269724 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5wjrb\"" Jul 02 23:23:05.396133 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.396092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.396314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.396171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/203ac940-1810-4a96-b963-d6145fc9de78-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.497348 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.497263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.497348 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.497322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/203ac940-1810-4a96-b963-d6145fc9de78-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.497867 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:05.497437 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:05.497867 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:05.497506 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:05.99749011 +0000 UTC m=+40.082091505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:05.498121 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.498099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/203ac940-1810-4a96-b963-d6145fc9de78-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:05.758349 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.758260 2562 generic.go:356] "Generic (PLEG): container finished" podID="ed8fda13-208f-47f1-bba0-e3f41fb34716" containerID="99bfd3c12557e649be345439949f974e1ff25196392466e5155bcb17fd41872a" exitCode=0 Jul 02 23:23:05.758482 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:05.758343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerDied","Data":"99bfd3c12557e649be345439949f974e1ff25196392466e5155bcb17fd41872a"} Jul 02 23:23:06.002070 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:06.002039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:06.002240 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.002196 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:06.002311 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.002276 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:07.002258626 +0000 UTC m=+41.086860022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:06.505737 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:06.505701 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:06.505790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.505880 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.505907 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.505919 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.505961 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:10.505940342 +0000 UTC m=+44.590541752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:06.506332 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:06.505981 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:10.505972633 +0000 UTC m=+44.590574031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:06.763929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:06.763852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k94b5" event={"ID":"ed8fda13-208f-47f1-bba0-e3f41fb34716","Type":"ContainerStarted","Data":"60f8543e4e00e3ffc2d0e04ff32f3990ad983f2ab7bc6d15e8c9c5252deb3e04"} Jul 02 23:23:06.788481 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:06.788436 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k94b5" podStartSLOduration=10.228693229 podStartE2EDuration="40.788421357s" totalImagesPullingTime="30.559728128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:06.787871862 +0000 UTC m=+40.872473282" watchObservedRunningTime="2026-07-02 23:23:06.788421357 +0000 UTC m=+40.873022778" Jul 02 23:23:07.010199 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:07.010167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:07.010341 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:07.010320 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:07.010398 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:07.010388 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:09.010372615 +0000 UTC m=+43.094974011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:07.768053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:07.767996 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtxgv" event={"ID":"cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f","Type":"ContainerStarted","Data":"0472e50cc971c52325da7b25d55739f21c14cfcb1d11e2a405c370a71583eb96"} Jul 02 23:23:07.782486 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:07.782425 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wtxgv" podStartSLOduration=3.118344278 podStartE2EDuration="5.782409251s" totalImagesPullingTime="2.664064973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:07.781726374 +0000 UTC m=+41.866327794" watchObservedRunningTime="2026-07-02 23:23:07.782409251 +0000 UTC m=+41.867010705" Jul 02 23:23:09.025075 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:09.025023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:09.025468 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:09.025171 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:09.025468 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:09.025234 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:13.025218769 +0000 UTC m=+47.109820165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:10.536763 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:10.536720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:10.536811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:10.536888 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:10.536918 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:10.536928 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:10.536973 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:18.53695215 +0000 UTC m=+52.621553550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:10.537251 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:10.536995 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:18.536985435 +0000 UTC m=+52.621586836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:12.465083 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.465051 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2cdcm"] Jul 02 23:23:12.499461 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.499419 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cdcm"] Jul 02 23:23:12.499620 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.499555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.502107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.502084 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:23:12.502107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.502102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:23:12.502385 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.502371 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:23:12.502432 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.502385 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-242f6\"" Jul 02 23:23:12.503182 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.503164 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:23:12.651720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.651679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-crio-socket\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.651948 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.651732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.651948 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.651758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfv7\" (UniqueName: \"kubernetes.io/projected/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-api-access-nwfv7\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.651948 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.651809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.651948 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.651856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-data-volume\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752219 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-crio-socket\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfv7\" (UniqueName: \"kubernetes.io/projected/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-api-access-nwfv7\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-data-volume\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752495 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:12.752373 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:12.752495 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752405 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-crio-socket\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.752495 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:12.752423 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:12.752495 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:12.752451 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:13.252431308 +0000 UTC m=+47.337032703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:12.752616 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:12.752498 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:13.252481526 +0000 UTC m=+47.337082934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : configmap "kube-rbac-proxy" not found Jul 02 23:23:12.752664 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.752649 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-data-volume\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:12.764953 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:12.764923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfv7\" (UniqueName: \"kubernetes.io/projected/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-api-access-nwfv7\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:13.054712 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:13.054636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:13.054852 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.054811 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:13.054902 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.054892 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:21.054876977 +0000 UTC m=+55.139478377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:13.255686 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:13.255650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:13.255686 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:13.255689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:13.255875 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.255797 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:13.255875 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.255803 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:13.255875 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.255858 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:14.255841663 +0000 UTC m=+48.340443058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:13.255875 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:13.255873 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:14.255865989 +0000 UTC m=+48.340467385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : configmap "kube-rbac-proxy" not found Jul 02 23:23:14.262448 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:14.262411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:14.262816 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:14.262456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:14.262816 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:14.262560 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:14.262816 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:14.262624 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:16.262609452 +0000 UTC m=+50.347210848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:14.262816 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:14.262562 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:14.262816 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:14.262694 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:16.262682885 +0000 UTC m=+50.347284280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : configmap "kube-rbac-proxy" not found Jul 02 23:23:14.371408 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:14.371377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hxd76_69b0b64a-23c1-45fb-84ba-9abd7c29442f/dns-node-resolver/0.log" Jul 02 23:23:15.771156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:15.771129 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r2fwr_cdd68c25-ded4-49cb-bf2a-d85ac74f44d3/node-ca/0.log" Jul 02 23:23:16.277881 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:16.277847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:16.278072 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:16.277891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:16.278072 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:16.277976 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:23:16.278072 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:16.277989 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:16.278072 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:16.278048 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:20.278034081 +0000 UTC m=+54.362635478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : configmap "kube-rbac-proxy" not found Jul 02 23:23:16.278072 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:16.278062 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:20.278056169 +0000 UTC m=+54.362657564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:16.571732 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:16.571665 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtxgv_cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f/serve-healthcheck-canary/0.log" Jul 02 23:23:18.596222 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:18.596178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:18.596259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:18.596342 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:18.596358 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74cd958458-ccx49: secret "image-registry-tls" not found Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:18.596376 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:18.596414 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls podName:13d7085f-e702-4b12-b222-7439cd6d2308 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:34.596397235 +0000 UTC m=+68.680998632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls") pod "image-registry-74cd958458-ccx49" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308") : secret "image-registry-tls" not found Jul 02 23:23:18.596647 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:18.596428 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls podName:a657f196-64df-4605-9d85-e726752bd25a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:34.596422817 +0000 UTC m=+68.681024213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls") pod "dns-default-2ckbs" (UID: "a657f196-64df-4605-9d85-e726752bd25a") : secret "dns-default-metrics-tls" not found Jul 02 23:23:20.309962 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:20.309918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:20.310408 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:20.309973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:20.310408 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:20.310090 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:23:20.310408 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:20.310157 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls podName:c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a nodeName:}" failed. No retries permitted until 2026-07-02 23:23:28.31014096 +0000 UTC m=+62.394742355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cdcm" (UID: "c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a") : secret "insights-runtime-extractor-tls" not found Jul 02 23:23:20.310522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:20.310499 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:21.117419 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:21.117363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:21.117599 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:21.117519 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:23:21.117599 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:21.117581 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert podName:203ac940-1810-4a96-b963-d6145fc9de78 nodeName:}" failed. No retries permitted until 2026-07-02 23:23:37.117565907 +0000 UTC m=+71.202167303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-ckp6s" (UID: "203ac940-1810-4a96-b963-d6145fc9de78") : secret "networking-console-plugin-cert" not found Jul 02 23:23:28.374921 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.374889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:28.377367 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.377344 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cdcm\" (UID: \"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a\") " pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:28.410978 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.410944 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-242f6\"" Jul 02 23:23:28.419330 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.419311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cdcm" Jul 02 23:23:28.543416 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.543379 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cdcm"] Jul 02 23:23:28.546315 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:28.546290 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc751b8b4_0cb8_4bc7_bc38_8cf4c722d02a.slice/crio-6dbd1face8afd469eedff67a1e8bbc6477c010ab94efb7e17bec86e9874824da WatchSource:0}: Error finding container 6dbd1face8afd469eedff67a1e8bbc6477c010ab94efb7e17bec86e9874824da: Status 404 returned error can't find the container with id 6dbd1face8afd469eedff67a1e8bbc6477c010ab94efb7e17bec86e9874824da Jul 02 23:23:28.733281 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.733259 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdq8z" Jul 02 23:23:28.819073 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:28.819039 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cdcm" event={"ID":"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a","Type":"ContainerStarted","Data":"6dbd1face8afd469eedff67a1e8bbc6477c010ab94efb7e17bec86e9874824da"} Jul 02 23:23:32.106933 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.106885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:32.109397 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.109364 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:23:32.120208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.120180 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba-original-pull-secret\") pod \"global-pull-secret-syncer-q5gzx\" (UID: \"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba\") " pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:32.207292 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.207264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:32.207408 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.207357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:32.209948 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.209930 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:23:32.210060 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.209974 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:23:32.220370 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.220346 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/795d14f0-d610-4fb2-9ccc-adc4f3478766-metrics-certs\") pod \"network-metrics-daemon-wq5cf\" (UID: \"795d14f0-d610-4fb2-9ccc-adc4f3478766\") " pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:32.220460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.220448 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:23:32.230904 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.230880 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wtm4\" (UniqueName: \"kubernetes.io/projected/0d35f306-64f6-47cb-bbfb-43ea24d281ff-kube-api-access-8wtm4\") pod \"network-check-target-jc2vq\" (UID: \"0d35f306-64f6-47cb-bbfb-43ea24d281ff\") " pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:32.341186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.341151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z4l4l\"" Jul 02 23:23:32.347280 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.347257 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p57lg\"" Jul 02 23:23:32.349017 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.348991 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wq5cf" Jul 02 23:23:32.350191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.350177 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q5gzx" Jul 02 23:23:32.355990 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.355973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:32.497103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.497059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q5gzx"] Jul 02 23:23:32.500668 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:32.500639 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694a6e24_4662_4aca_b3f8_bc4e9e3ae9ba.slice/crio-efe3573e563bbf2624669380201c95b9245351dd5a57eaede4517b000c2ae1e4 WatchSource:0}: Error finding container efe3573e563bbf2624669380201c95b9245351dd5a57eaede4517b000c2ae1e4: Status 404 returned error can't find the container with id efe3573e563bbf2624669380201c95b9245351dd5a57eaede4517b000c2ae1e4 Jul 02 23:23:32.710918 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.710766 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jc2vq"] Jul 02 23:23:32.713324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.713294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wq5cf"] Jul 02 23:23:32.713672 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:32.713649 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d35f306_64f6_47cb_bbfb_43ea24d281ff.slice/crio-a13cd08c558c378949c27a42f4a3ee2c6c70f6bbcc39ec545ed0abaadd9128c2 WatchSource:0}: Error finding container a13cd08c558c378949c27a42f4a3ee2c6c70f6bbcc39ec545ed0abaadd9128c2: Status 404 returned error can't find the container with id a13cd08c558c378949c27a42f4a3ee2c6c70f6bbcc39ec545ed0abaadd9128c2 Jul 02 23:23:32.716459 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:32.716439 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795d14f0_d610_4fb2_9ccc_adc4f3478766.slice/crio-65e9d3b919277273dd502fdf84ed4712e0eb13cb21fdc0d37d73385b47b364e7 WatchSource:0}: Error finding container 65e9d3b919277273dd502fdf84ed4712e0eb13cb21fdc0d37d73385b47b364e7: Status 404 returned error can't find the container with id 65e9d3b919277273dd502fdf84ed4712e0eb13cb21fdc0d37d73385b47b364e7 Jul 02 23:23:32.830804 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.830775 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jc2vq" event={"ID":"0d35f306-64f6-47cb-bbfb-43ea24d281ff","Type":"ContainerStarted","Data":"a13cd08c558c378949c27a42f4a3ee2c6c70f6bbcc39ec545ed0abaadd9128c2"} Jul 02 23:23:32.832044 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.831996 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q5gzx" event={"ID":"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba","Type":"ContainerStarted","Data":"efe3573e563bbf2624669380201c95b9245351dd5a57eaede4517b000c2ae1e4"} Jul 02 23:23:32.833525 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.833504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cdcm" event={"ID":"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a","Type":"ContainerStarted","Data":"8f1878a6229a3e8799517c600ed929f84c35d3ea315cb3ed0fead5e232d4a1fe"} Jul 02 23:23:32.833611 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.833551 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cdcm" event={"ID":"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a","Type":"ContainerStarted","Data":"b2979f81b5e904814c2489894e9eceb45b861406eef68a4ab9845f60aba91e28"} Jul 02 23:23:32.834589 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:32.834569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wq5cf" event={"ID":"795d14f0-d610-4fb2-9ccc-adc4f3478766","Type":"ContainerStarted","Data":"65e9d3b919277273dd502fdf84ed4712e0eb13cb21fdc0d37d73385b47b364e7"} Jul 02 23:23:34.521562 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.521528 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv"] Jul 02 23:23:34.539185 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.539156 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74cd958458-ccx49"] Jul 02 23:23:34.539342 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.539206 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9"] Jul 02 23:23:34.540428 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.540409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.540741 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:34.540708 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-74cd958458-ccx49" podUID="13d7085f-e702-4b12-b222-7439cd6d2308" Jul 02 23:23:34.543900 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.543876 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 23:23:34.543992 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.543933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 23:23:34.544172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.544156 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 23:23:34.544262 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.544248 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 23:23:34.555217 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.555196 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv"] Jul 02 23:23:34.555319 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.555241 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9"] Jul 02 23:23:34.555355 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.555336 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.558083 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.558062 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 23:23:34.558328 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.558313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 23:23:34.562042 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.559021 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 23:23:34.562042 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.559361 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 23:23:34.625938 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.625902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:34.626117 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.625969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:34.626117 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6wtb\" (UniqueName: \"kubernetes.io/projected/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-kube-api-access-n6wtb\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.626117 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.626117 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626101 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a987a0-9ab2-491d-9d1e-5985209c429f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlfrl\" (UniqueName: \"kubernetes.io/projected/b5a987a0-9ab2-491d-9d1e-5985209c429f-kube-api-access-vlfrl\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-tmp\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-klusterlet-config\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.626324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.626266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.629205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.629156 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"image-registry-74cd958458-ccx49\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:34.641885 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.641834 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a657f196-64df-4605-9d85-e726752bd25a-metrics-tls\") pod \"dns-default-2ckbs\" (UID: \"a657f196-64df-4605-9d85-e726752bd25a\") " pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:34.726650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6wtb\" (UniqueName: \"kubernetes.io/projected/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-kube-api-access-n6wtb\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a987a0-9ab2-491d-9d1e-5985209c429f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlfrl\" (UniqueName: \"kubernetes.io/projected/b5a987a0-9ab2-491d-9d1e-5985209c429f-kube-api-access-vlfrl\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-tmp\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-klusterlet-config\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.726855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.726855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.727401 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.727344 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-tmp\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.727739 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.727691 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a987a0-9ab2-491d-9d1e-5985209c429f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.730969 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.730939 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.731138 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.731104 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.731228 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.731202 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-hub\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.732603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.732560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-klusterlet-config\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.733265 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.732911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b5a987a0-9ab2-491d-9d1e-5985209c429f-ca\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.735780 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.735752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6wtb\" (UniqueName: \"kubernetes.io/projected/b13be81d-3e79-4580-8e9e-0dc08a8f0b86-kube-api-access-n6wtb\") pod \"klusterlet-addon-workmgr-f94bcbd66-b2slv\" (UID: \"b13be81d-3e79-4580-8e9e-0dc08a8f0b86\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.736881 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.736854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlfrl\" (UniqueName: \"kubernetes.io/projected/b5a987a0-9ab2-491d-9d1e-5985209c429f-kube-api-access-vlfrl\") pod \"cluster-proxy-proxy-agent-566b8695c5-z26g9\" (UID: \"b5a987a0-9ab2-491d-9d1e-5985209c429f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.843887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.843815 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:34.844075 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.844043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wq5cf" event={"ID":"795d14f0-d610-4fb2-9ccc-adc4f3478766","Type":"ContainerStarted","Data":"5cadc6f32643ce0d33734064bd6470486a6414ed99b8ccb63fa0906a03929b6d"} Jul 02 23:23:34.844168 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.844129 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wq5cf" event={"ID":"795d14f0-d610-4fb2-9ccc-adc4f3478766","Type":"ContainerStarted","Data":"f67f1df3f78c85500ac8e9c7f9b727f651fc07bd133d32e9a6295045809578b5"} Jul 02 23:23:34.855179 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.855106 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:34.873648 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.873594 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qpvbp\"" Jul 02 23:23:34.876703 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.876680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" Jul 02 23:23:34.880492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.880460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:34.881697 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.881661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:34.883564 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.883518 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wq5cf" podStartSLOduration=66.775840689 podStartE2EDuration="1m7.883500593s" totalImagesPullingTime="1.107659904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:34.883487133 +0000 UTC m=+68.968088552" watchObservedRunningTime="2026-07-02 23:23:34.883500593 +0000 UTC m=+68.968102012" Jul 02 23:23:34.928727 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928700 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.928845 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928744 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.928845 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928770 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.928845 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928798 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.929029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928844 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.929029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928874 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.929029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cppdb\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.929029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.928955 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca\") pod \"13d7085f-e702-4b12-b222-7439cd6d2308\" (UID: \"13d7085f-e702-4b12-b222-7439cd6d2308\") " Jul 02 23:23:34.929476 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.929449 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:23:34.929476 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.929436 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:23:34.929833 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.929798 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:23:34.932441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.932384 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:23:34.932669 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.932631 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb" (OuterVolumeSpecName: "kube-api-access-cppdb") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "kube-api-access-cppdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:23:34.933132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.933112 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:23:34.934492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.934473 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:23:34.934754 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:34.934727 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "13d7085f-e702-4b12-b222-7439cd6d2308" (UID: "13d7085f-e702-4b12-b222-7439cd6d2308"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:23:35.030636 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030594 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-registry-certificates\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030636 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030629 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cppdb\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-kube-api-access-cppdb\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030636 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030640 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13d7085f-e702-4b12-b222-7439cd6d2308-trusted-ca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030652 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-installation-pull-secrets\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030666 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-registry-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030678 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13d7085f-e702-4b12-b222-7439cd6d2308-bound-sa-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030691 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13d7085f-e702-4b12-b222-7439cd6d2308-ca-trust-extracted\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.030905 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.030702 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13d7085f-e702-4b12-b222-7439cd6d2308-image-registry-private-configuration\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:23:35.846341 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.846310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74cd958458-ccx49" Jul 02 23:23:35.879277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.879243 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74cd958458-ccx49"] Jul 02 23:23:35.882832 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:35.882757 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74cd958458-ccx49"] Jul 02 23:23:36.430480 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:36.430442 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d7085f-e702-4b12-b222-7439cd6d2308" path="/var/lib/kubelet/pods/13d7085f-e702-4b12-b222-7439cd6d2308/volumes" Jul 02 23:23:37.146438 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:37.146404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:37.149086 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:37.149061 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/203ac940-1810-4a96-b963-d6145fc9de78-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-ckp6s\" (UID: \"203ac940-1810-4a96-b963-d6145fc9de78\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:37.379652 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:37.379624 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5wjrb\"" Jul 02 23:23:37.387696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:37.387671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" Jul 02 23:23:38.570909 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.570814 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s"] Jul 02 23:23:38.577834 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:38.577791 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203ac940_1810_4a96_b963_d6145fc9de78.slice/crio-4a660c01b5498c6c98cdb22672d548ca6306f1c2b880fd5cbc91500b2beda02e WatchSource:0}: Error finding container 4a660c01b5498c6c98cdb22672d548ca6306f1c2b880fd5cbc91500b2beda02e: Status 404 returned error can't find the container with id 4a660c01b5498c6c98cdb22672d548ca6306f1c2b880fd5cbc91500b2beda02e Jul 02 23:23:38.594723 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.594676 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9"] Jul 02 23:23:38.598877 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:38.598854 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5a987a0_9ab2_491d_9d1e_5985209c429f.slice/crio-3cf96d686f4c990c03d3687ea334f6672047a8a34225b57b5b488d4b9396b58f WatchSource:0}: Error finding container 3cf96d686f4c990c03d3687ea334f6672047a8a34225b57b5b488d4b9396b58f: Status 404 returned error can't find the container with id 3cf96d686f4c990c03d3687ea334f6672047a8a34225b57b5b488d4b9396b58f Jul 02 23:23:38.611645 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.611518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ckbs"] Jul 02 23:23:38.614025 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:38.613975 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda657f196_64df_4605_9d85_e726752bd25a.slice/crio-1f2b748351ddd28470014e126e8d2dcd1f72752053f75d7e13298333660ef618 WatchSource:0}: Error finding container 1f2b748351ddd28470014e126e8d2dcd1f72752053f75d7e13298333660ef618: Status 404 returned error can't find the container with id 1f2b748351ddd28470014e126e8d2dcd1f72752053f75d7e13298333660ef618 Jul 02 23:23:38.811116 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.811089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv"] Jul 02 23:23:38.814364 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:38.814338 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13be81d_3e79_4580_8e9e_0dc08a8f0b86.slice/crio-38d1e2be966ab963af585eab42be5c1e56c5f3e5be73fe093c4611b31f35e691 WatchSource:0}: Error finding container 38d1e2be966ab963af585eab42be5c1e56c5f3e5be73fe093c4611b31f35e691: Status 404 returned error can't find the container with id 38d1e2be966ab963af585eab42be5c1e56c5f3e5be73fe093c4611b31f35e691 Jul 02 23:23:38.857161 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.857125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jc2vq" event={"ID":"0d35f306-64f6-47cb-bbfb-43ea24d281ff","Type":"ContainerStarted","Data":"d162c45c61cd76564cb24f09188743537c35191cfc0c35920ca3352234759cfe"} Jul 02 23:23:38.857319 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.857223 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:23:38.858596 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.858572 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q5gzx" event={"ID":"694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba","Type":"ContainerStarted","Data":"3d4ae9109385bddb606ae68ef840adcd4a0e8a797f975b3b1f06d46504e89b3c"} Jul 02 23:23:38.859558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.859539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" event={"ID":"203ac940-1810-4a96-b963-d6145fc9de78","Type":"ContainerStarted","Data":"4a660c01b5498c6c98cdb22672d548ca6306f1c2b880fd5cbc91500b2beda02e"} Jul 02 23:23:38.860621 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.860598 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" event={"ID":"b5a987a0-9ab2-491d-9d1e-5985209c429f","Type":"ContainerStarted","Data":"3cf96d686f4c990c03d3687ea334f6672047a8a34225b57b5b488d4b9396b58f"} Jul 02 23:23:38.861675 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.861625 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" event={"ID":"b13be81d-3e79-4580-8e9e-0dc08a8f0b86","Type":"ContainerStarted","Data":"38d1e2be966ab963af585eab42be5c1e56c5f3e5be73fe093c4611b31f35e691"} Jul 02 23:23:38.863537 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.863518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cdcm" event={"ID":"c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a","Type":"ContainerStarted","Data":"fc59d09ec50403c5c5ce0157dd02e5ac4782b03ccd0d349ad9e76881a0bddf8e"} Jul 02 23:23:38.864427 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.864411 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ckbs" event={"ID":"a657f196-64df-4605-9d85-e726752bd25a","Type":"ContainerStarted","Data":"1f2b748351ddd28470014e126e8d2dcd1f72752053f75d7e13298333660ef618"} Jul 02 23:23:38.871527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.871491 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jc2vq" podStartSLOduration=66.189662498 podStartE2EDuration="1m11.871479378s" totalImagesPullingTime="5.68181688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:38.871046977 +0000 UTC m=+72.955648396" watchObservedRunningTime="2026-07-02 23:23:38.871479378 +0000 UTC m=+72.956080796" Jul 02 23:23:38.886708 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.886659 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2cdcm" podStartSLOduration=17.168020823 podStartE2EDuration="26.886646195s" totalImagesPullingTime="9.718625372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:23:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:38.88581484 +0000 UTC m=+72.970416273" watchObservedRunningTime="2026-07-02 23:23:38.886646195 +0000 UTC m=+72.971247613" Jul 02 23:23:38.899326 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:38.899287 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q5gzx" podStartSLOduration=66.005105656 podStartE2EDuration="1m11.899275374s" totalImagesPullingTime="5.894169718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:38.899185756 +0000 UTC m=+72.983787173" watchObservedRunningTime="2026-07-02 23:23:38.899275374 +0000 UTC m=+72.983876791" Jul 02 23:23:44.890123 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.890065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" event={"ID":"b13be81d-3e79-4580-8e9e-0dc08a8f0b86","Type":"ContainerStarted","Data":"9d398eb0e9783f010c1340b4cb76721f58a90aa799cf4cc8f8a372a49023a2f9"} Jul 02 23:23:44.890506 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.890175 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:44.892083 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.892060 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" Jul 02 23:23:44.892277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.892256 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ckbs" event={"ID":"a657f196-64df-4605-9d85-e726752bd25a","Type":"ContainerStarted","Data":"1923008b3973ec3ae1bbd7e5f6296bb5f723b4873e710b4b9a1c0a236a320069"} Jul 02 23:23:44.892327 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.892301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ckbs" event={"ID":"a657f196-64df-4605-9d85-e726752bd25a","Type":"ContainerStarted","Data":"01e12df0e0c3581bb1b03fa21ae9e4dd7f94a2f7d633882f488c4c4ad677cdc2"} Jul 02 23:23:44.892371 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.892327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:44.893798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.893777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" event={"ID":"203ac940-1810-4a96-b963-d6145fc9de78","Type":"ContainerStarted","Data":"5744f665ba0283b3fcfcb36f633653b4ca8225fde6f29f2211da7d69fa8a1ea8"} Jul 02 23:23:44.895780 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.895762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" event={"ID":"b5a987a0-9ab2-491d-9d1e-5985209c429f","Type":"ContainerStarted","Data":"6e212cee0265f428b52066b3a21b1e7ae15954f38a3b702925862619ae596e45"} Jul 02 23:23:44.895877 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.895787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" event={"ID":"b5a987a0-9ab2-491d-9d1e-5985209c429f","Type":"ContainerStarted","Data":"80fafd2ad04fcec2fa02e82ac0bd2bea77edb9b8eb8d781162210ea09da45fa2"} Jul 02 23:23:44.895877 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.895800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" event={"ID":"b5a987a0-9ab2-491d-9d1e-5985209c429f","Type":"ContainerStarted","Data":"adf7493e60ca9dc935ec530569d2b22917247f59ca8f043b321ba45297e8a639"} Jul 02 23:23:44.904847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.904802 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f94bcbd66-b2slv" podStartSLOduration=5.476086417 podStartE2EDuration="10.904789139s" totalImagesPullingTime="5.428702722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.904409866 +0000 UTC m=+78.989011284" watchObservedRunningTime="2026-07-02 23:23:44.904789139 +0000 UTC m=+78.989390557" Jul 02 23:23:44.920129 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.920085 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2ckbs" podStartSLOduration=37.379575005 podStartE2EDuration="42.920072872s" totalImagesPullingTime="5.540497867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.919071063 +0000 UTC m=+79.003672481" watchObservedRunningTime="2026-07-02 23:23:44.920072872 +0000 UTC m=+79.004674289" Jul 02 23:23:44.936217 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.936173 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566b8695c5-z26g9" podStartSLOduration=5.295107403 podStartE2EDuration="10.936161741s" totalImagesPullingTime="5.641054338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.934992592 +0000 UTC m=+79.019594011" watchObservedRunningTime="2026-07-02 23:23:44.936161741 +0000 UTC m=+79.020763156" Jul 02 23:23:44.967406 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:44.967360 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-ckp6s" podStartSLOduration=34.392483242 podStartE2EDuration="39.967347021s" totalImagesPullingTime="5.574863779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:44.966634638 +0000 UTC m=+79.051236058" watchObservedRunningTime="2026-07-02 23:23:44.967347021 +0000 UTC m=+79.051948478" Jul 02 23:23:50.689258 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.689142 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qqhql"] Jul 02 23:23:50.693376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.693357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.695989 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.695967 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:23:50.696219 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.696202 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:23:50.696281 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.696202 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:23:50.697101 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.697082 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:23:50.697245 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.697228 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sgptq\"" Jul 02 23:23:50.697340 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.697299 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:23:50.697340 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.697333 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:23:50.747138 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-sys\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747138 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-wtmp\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-textfile\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-metrics-client-ca\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-root\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747369 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll6bx\" (UniqueName: \"kubernetes.io/projected/e6675f38-fa06-4f58-bd0b-3934250ded1b-kube-api-access-ll6bx\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.747553 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.747381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-tls\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-textfile\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-metrics-client-ca\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-root\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll6bx\" (UniqueName: \"kubernetes.io/projected/e6675f38-fa06-4f58-bd0b-3934250ded1b-kube-api-access-ll6bx\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-tls\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848363 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-root\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-sys\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848429 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-sys\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-textfile\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-wtmp\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-wtmp\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848938 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-accelerators-collector-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.848973 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.848937 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6675f38-fa06-4f58-bd0b-3934250ded1b-metrics-client-ca\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.850832 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.850813 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.850939 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.850923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6675f38-fa06-4f58-bd0b-3934250ded1b-node-exporter-tls\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:50.857122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:50.857096 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll6bx\" (UniqueName: \"kubernetes.io/projected/e6675f38-fa06-4f58-bd0b-3934250ded1b-kube-api-access-ll6bx\") pod \"node-exporter-qqhql\" (UID: \"e6675f38-fa06-4f58-bd0b-3934250ded1b\") " pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:51.002975 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:51.002945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qqhql" Jul 02 23:23:51.013375 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:51.013346 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6675f38_fa06_4f58_bd0b_3934250ded1b.slice/crio-2c60f328b2420aa77c92e62f0b0c640478c808ae3ae04e9fa01f1c4b84d63cd3 WatchSource:0}: Error finding container 2c60f328b2420aa77c92e62f0b0c640478c808ae3ae04e9fa01f1c4b84d63cd3: Status 404 returned error can't find the container with id 2c60f328b2420aa77c92e62f0b0c640478c808ae3ae04e9fa01f1c4b84d63cd3 Jul 02 23:23:51.921291 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:51.921259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qqhql" event={"ID":"e6675f38-fa06-4f58-bd0b-3934250ded1b","Type":"ContainerStarted","Data":"2c60f328b2420aa77c92e62f0b0c640478c808ae3ae04e9fa01f1c4b84d63cd3"} Jul 02 23:23:52.925784 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:52.925738 2562 generic.go:356] "Generic (PLEG): container finished" podID="e6675f38-fa06-4f58-bd0b-3934250ded1b" containerID="aaa857dde01e8bd454e0de42fe8662d038780c13ec60b54c6a96914e2948ac53" exitCode=0 Jul 02 23:23:52.926157 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:52.925825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qqhql" event={"ID":"e6675f38-fa06-4f58-bd0b-3934250ded1b","Type":"ContainerDied","Data":"aaa857dde01e8bd454e0de42fe8662d038780c13ec60b54c6a96914e2948ac53"} Jul 02 23:23:53.931326 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:53.931285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qqhql" event={"ID":"e6675f38-fa06-4f58-bd0b-3934250ded1b","Type":"ContainerStarted","Data":"7e222a3e455480bb403485689afd68279d884d3a7a81219216c59171a3cb280d"} Jul 02 23:23:53.931698 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:53.931348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qqhql" event={"ID":"e6675f38-fa06-4f58-bd0b-3934250ded1b","Type":"ContainerStarted","Data":"f3401599584b1e5d41a6e226bdc223b466538b1cbfda6edb8820f7bed023d7cf"} Jul 02 23:23:53.955382 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:53.955328 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qqhql" podStartSLOduration=2.948617953 podStartE2EDuration="3.955311753s" totalImagesPullingTime="935.439945ms" totalInitContainerRuntime="71.253855ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:53.954370822 +0000 UTC m=+88.038972243" watchObservedRunningTime="2026-07-02 23:23:53.955311753 +0000 UTC m=+88.039913166" Jul 02 23:23:54.901866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:54.901837 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2ckbs" Jul 02 23:23:55.410800 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.410765 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz"] Jul 02 23:23:55.414124 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.414103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:55.416635 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.416610 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 23:23:55.416723 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.416617 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bbhps\"" Jul 02 23:23:55.420467 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.420272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz"] Jul 02 23:23:55.480527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.480493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:55.480527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.480525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bfrn\" (UniqueName: \"kubernetes.io/projected/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-kube-api-access-9bfrn\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:55.581789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.581747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:55.581789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.581796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bfrn\" (UniqueName: \"kubernetes.io/projected/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-kube-api-access-9bfrn\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:55.581986 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:55.581900 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 02 23:23:55.581986 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:23:55.581961 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert podName:2f9aec65-bd86-4e1a-913c-032fcc6e6cab nodeName:}" failed. No retries permitted until 2026-07-02 23:23:56.081944144 +0000 UTC m=+90.166545543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-bqghz" (UID: "2f9aec65-bd86-4e1a-913c-032fcc6e6cab") : secret "monitoring-plugin-cert" not found Jul 02 23:23:55.590646 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:55.590625 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bfrn\" (UniqueName: \"kubernetes.io/projected/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-kube-api-access-9bfrn\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:56.085274 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.085236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:56.087744 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.087723 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2f9aec65-bd86-4e1a-913c-032fcc6e6cab-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bqghz\" (UID: \"2f9aec65-bd86-4e1a-913c-032fcc6e6cab\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:56.323465 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.323433 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:56.452349 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.452316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz"] Jul 02 23:23:56.455428 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:56.455405 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9aec65_bd86_4e1a_913c_032fcc6e6cab.slice/crio-c8a305ba78da133471d6214724fc23eca8cd31bee1709d50974005f05a415925 WatchSource:0}: Error finding container c8a305ba78da133471d6214724fc23eca8cd31bee1709d50974005f05a415925: Status 404 returned error can't find the container with id c8a305ba78da133471d6214724fc23eca8cd31bee1709d50974005f05a415925 Jul 02 23:23:56.829752 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.829721 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:23:56.834302 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.834284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.837058 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.837036 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 23:23:56.837172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.837081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nbjlx\"" Jul 02 23:23:56.837172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.837093 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 23:23:56.837172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.837081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 23:23:56.837327 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.837036 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-85olob94i7dom\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838418 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838427 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838356 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838428 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 23:23:56.838544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.838364 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 23:23:56.839618 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.839598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 23:23:56.839744 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.839621 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 23:23:56.841188 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.841169 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 23:23:56.844255 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.844237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 23:23:56.850506 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.850483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:23:56.892350 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892350 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khvvt\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.892829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.892829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.943453 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.943419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" event={"ID":"2f9aec65-bd86-4e1a-913c-032fcc6e6cab","Type":"ContainerStarted","Data":"c8a305ba78da133471d6214724fc23eca8cd31bee1709d50974005f05a415925"} Jul 02 23:23:56.993473 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993473 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.993929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994372 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994372 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.993986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994372 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khvvt\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994372 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994372 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994359 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994602 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.994659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994643 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.995378 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.994874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.996963 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.996925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.997654 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.997631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998347 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.997889 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998347 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.997718 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998347 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.998159 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998347 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.996943 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.998571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998661 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.998642 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.998885 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.998832 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.999534 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.999512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:56.999715 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:56.999701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.000755 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.000728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.001088 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.001068 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.001142 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.001126 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.001266 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.001247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.004828 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.004803 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khvvt\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt\") pod \"prometheus-k8s-0\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.144666 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.144580 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:23:57.301954 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.301918 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:23:57.304805 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:23:57.304774 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e4e68e_0aaa_4afd_b121_314893af9730.slice/crio-01743b4089ac40070045df5c652840e8b1f9aeee24adc3a918aea06b1f79ebbf WatchSource:0}: Error finding container 01743b4089ac40070045df5c652840e8b1f9aeee24adc3a918aea06b1f79ebbf: Status 404 returned error can't find the container with id 01743b4089ac40070045df5c652840e8b1f9aeee24adc3a918aea06b1f79ebbf Jul 02 23:23:57.948638 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.948582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"01743b4089ac40070045df5c652840e8b1f9aeee24adc3a918aea06b1f79ebbf"} Jul 02 23:23:57.950312 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.950282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" event={"ID":"2f9aec65-bd86-4e1a-913c-032fcc6e6cab","Type":"ContainerStarted","Data":"c8932f80ee0f758326aabbab03a50f17538a914fa3ba21db7dd0e1ce23660b82"} Jul 02 23:23:57.950540 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.950522 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:57.956212 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.956190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" Jul 02 23:23:57.964398 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:57.964346 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bqghz" podStartSLOduration=1.642965832 podStartE2EDuration="2.964330617s" totalImagesPullingTime="1.321364785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:23:57.963770914 +0000 UTC m=+92.048372332" watchObservedRunningTime="2026-07-02 23:23:57.964330617 +0000 UTC m=+92.048932039" Jul 02 23:23:58.954882 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:58.954850 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041" exitCode=0 Jul 02 23:23:58.955383 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:23:58.954943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041"} Jul 02 23:24:02.972409 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:02.972367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb"} Jul 02 23:24:02.972872 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:02.972454 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509"} Jul 02 23:24:04.982165 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:04.982126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7"} Jul 02 23:24:04.982547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:04.982187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77"} Jul 02 23:24:04.982547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:04.982213 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae"} Jul 02 23:24:04.982547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:04.982236 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerStarted","Data":"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81"} Jul 02 23:24:05.011498 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:05.011438 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.783329442 podStartE2EDuration="9.011420211s" totalImagesPullingTime="6.208040399s" totalInitContainerRuntime="20.05037ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:23:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:24:05.009458808 +0000 UTC m=+99.094060237" watchObservedRunningTime="2026-07-02 23:24:05.011420211 +0000 UTC m=+99.096021700" Jul 02 23:24:07.145029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:07.144966 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:24:09.873305 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:09.873270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jc2vq" Jul 02 23:24:57.145312 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:57.145272 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:24:57.164654 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:57.164629 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:24:57.179462 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:24:57.179438 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:15.175581 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175544 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:15.176134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175831 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="prometheus" containerID="cri-o://b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509" gracePeriod=600 Jul 02 23:25:15.176134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175880 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7" gracePeriod=600 Jul 02 23:25:15.176134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175909 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="config-reloader" containerID="cri-o://0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb" gracePeriod=600 Jul 02 23:25:15.176134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175859 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy" containerID="cri-o://8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77" gracePeriod=600 Jul 02 23:25:15.176134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.175901 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="thanos-sidecar" containerID="cri-o://048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81" gracePeriod=600 Jul 02 23:25:15.176386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:15.176139 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-web" containerID="cri-o://e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae" gracePeriod=600 Jul 02 23:25:16.236147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236113 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7" exitCode=0 Jul 02 23:25:16.236147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236143 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77" exitCode=0 Jul 02 23:25:16.236147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236149 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81" exitCode=0 Jul 02 23:25:16.236147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236155 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb" exitCode=0 Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236165 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509" exitCode=0 Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7"} Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77"} Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81"} Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236306 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb"} Jul 02 23:25:16.236604 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.236329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509"} Jul 02 23:25:16.419427 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.419401 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:16.484906 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484833 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.484906 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484870 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.484906 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484888 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484918 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484945 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484966 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.484998 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485055 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485077 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485104 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485130 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485161 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485190 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485222 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485240 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485260 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485274 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khvvt\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485275 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485337 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485405 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle\") pod \"d2e4e68e-0aaa-4afd-b121-314893af9730\" (UID: \"d2e4e68e-0aaa-4afd-b121-314893af9730\") " Jul 02 23:25:16.485695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485623 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.486196 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485799 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.486196 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.485820 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.486196 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.486166 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.486541 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.486479 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.487265 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.487143 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:25:16.487925 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.487570 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.487925 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.487578 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.487925 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.487893 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:25:16.490917 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.490892 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.490917 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.490909 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.491105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.490980 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out" (OuterVolumeSpecName: "config-out") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:25:16.491105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491070 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:25:16.491276 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491253 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.491378 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491358 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.491433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491408 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.491492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491452 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.491890 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.491861 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config" (OuterVolumeSpecName: "config") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.492392 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.492372 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.492754 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.492729 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt" (OuterVolumeSpecName: "kube-api-access-khvvt") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "kube-api-access-khvvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:25:16.500178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.500152 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config" (OuterVolumeSpecName: "web-config") pod "d2e4e68e-0aaa-4afd-b121-314893af9730" (UID: "d2e4e68e-0aaa-4afd-b121-314893af9730"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:25:16.586681 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586647 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-kube-rbac-proxy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586681 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586675 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-metrics-client-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586681 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586685 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586697 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-web-config\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586707 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586719 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586727 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-config-out\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586735 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586743 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-db\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586751 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586759 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-tls-assets\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586767 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586775 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-secret-grpc-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586784 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-metrics-client-ca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586792 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586800 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2e4e68e-0aaa-4afd-b121-314893af9730-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586808 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2e4e68e-0aaa-4afd-b121-314893af9730-config\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:16.586898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:16.586815 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khvvt\" (UniqueName: \"kubernetes.io/projected/d2e4e68e-0aaa-4afd-b121-314893af9730-kube-api-access-khvvt\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:25:17.242352 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.242323 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerID="e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae" exitCode=0 Jul 02 23:25:17.242717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.242395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae"} Jul 02 23:25:17.242717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.242404 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.242717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.242450 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2e4e68e-0aaa-4afd-b121-314893af9730","Type":"ContainerDied","Data":"01743b4089ac40070045df5c652840e8b1f9aeee24adc3a918aea06b1f79ebbf"} Jul 02 23:25:17.242717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.242482 2562 scope.go:117] "RemoveContainer" containerID="1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7" Jul 02 23:25:17.251164 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.251144 2562 scope.go:117] "RemoveContainer" containerID="8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77" Jul 02 23:25:17.259848 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.259815 2562 scope.go:117] "RemoveContainer" containerID="e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae" Jul 02 23:25:17.265618 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.265598 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:17.268160 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.268146 2562 scope.go:117] "RemoveContainer" containerID="048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81" Jul 02 23:25:17.272574 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.272555 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:17.275059 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.274980 2562 scope.go:117] "RemoveContainer" containerID="0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb" Jul 02 23:25:17.281406 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.281391 2562 scope.go:117] "RemoveContainer" containerID="b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509" Jul 02 23:25:17.288506 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.288488 2562 scope.go:117] "RemoveContainer" containerID="0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041" Jul 02 23:25:17.295273 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295159 2562 scope.go:117] "RemoveContainer" containerID="1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7" Jul 02 23:25:17.295456 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.295431 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7\": container with ID starting with 1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7 not found: ID does not exist" containerID="1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7" Jul 02 23:25:17.295511 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295469 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7"} err="failed to get container status \"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7\": rpc error: code = NotFound desc = could not find container \"1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7\": container with ID starting with 1372dc491127083f49b6fbdf997b814ccecca838da0478e9d578671be82f12e7 not found: ID does not exist" Jul 02 23:25:17.295553 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295515 2562 scope.go:117] "RemoveContainer" containerID="8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77" Jul 02 23:25:17.295585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295548 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:17.295764 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.295746 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77\": container with ID starting with 8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77 not found: ID does not exist" containerID="8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77" Jul 02 23:25:17.295837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295764 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="config-reloader" Jul 02 23:25:17.295837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295774 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="config-reloader" Jul 02 23:25:17.295837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295783 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-web" Jul 02 23:25:17.295837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295773 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77"} err="failed to get container status \"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77\": rpc error: code = NotFound desc = could not find container \"8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77\": container with ID starting with 8c68e75eba6a18d9e2b1406c9d84a63ac792f948f1fabfc59b74f8c01c409a77 not found: ID does not exist" Jul 02 23:25:17.295837 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295797 2562 scope.go:117] "RemoveContainer" containerID="e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295788 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-web" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295885 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="init-config-reloader" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295898 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="init-config-reloader" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295908 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="thanos-sidecar" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295917 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="thanos-sidecar" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295927 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-thanos" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295937 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-thanos" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295951 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="prometheus" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295959 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="prometheus" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295969 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.295977 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.296022 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae\": container with ID starting with e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae not found: ID does not exist" containerID="e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296042 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae"} err="failed to get container status \"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae\": rpc error: code = NotFound desc = could not find container \"e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae\": container with ID starting with e706670d333142a7b810a4e0d06ce845ed24ce355ca8d4eadcc4fe540b4c0bae not found: ID does not exist" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296058 2562 scope.go:117] "RemoveContainer" containerID="048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296074 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296086 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-thanos" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296097 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="thanos-sidecar" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296108 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="kube-rbac-proxy-web" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296116 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="prometheus" Jul 02 23:25:17.296113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296124 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" containerName="config-reloader" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.296288 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81\": container with ID starting with 048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81 not found: ID does not exist" containerID="048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296304 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81"} err="failed to get container status \"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81\": rpc error: code = NotFound desc = could not find container \"048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81\": container with ID starting with 048075d361a120105a16a25f4504c4abf74d60de1cb555d2d3d111235ef04c81 not found: ID does not exist" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296319 2562 scope.go:117] "RemoveContainer" containerID="0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.296518 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb\": container with ID starting with 0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb not found: ID does not exist" containerID="0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296536 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb"} err="failed to get container status \"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb\": rpc error: code = NotFound desc = could not find container \"0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb\": container with ID starting with 0ea1b40a0afc91a893458b502db41c6808ea0c6655ff5720a410358acac5becb not found: ID does not exist" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296548 2562 scope.go:117] "RemoveContainer" containerID="b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.296707 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509\": container with ID starting with b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509 not found: ID does not exist" containerID="b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296722 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509"} err="failed to get container status \"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509\": rpc error: code = NotFound desc = could not find container \"b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509\": container with ID starting with b081d445e2aa6ecf4191bb2ca7a09bc78376c507b7b3a0223cad9831a7526509 not found: ID does not exist" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296735 2562 scope.go:117] "RemoveContainer" containerID="0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:25:17.296912 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041\": container with ID starting with 0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041 not found: ID does not exist" containerID="0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041" Jul 02 23:25:17.297029 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.296930 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041"} err="failed to get container status \"0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041\": rpc error: code = NotFound desc = could not find container \"0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041\": container with ID starting with 0c7ee62f35e366abfc339f9a75e8d5ba47f496ac08755af69b213768ab786041 not found: ID does not exist" Jul 02 23:25:17.301476 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.301459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.304156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304134 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 23:25:17.304243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304162 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 23:25:17.304243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304214 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nbjlx\"" Jul 02 23:25:17.304402 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304328 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 23:25:17.304514 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304476 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-85olob94i7dom\"" Jul 02 23:25:17.304608 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304538 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 23:25:17.304684 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304619 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 23:25:17.304753 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 23:25:17.304753 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304739 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 23:25:17.304868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 23:25:17.304868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 23:25:17.304868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304771 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 23:25:17.304868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304789 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 23:25:17.304868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.304856 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 23:25:17.305441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.305426 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 23:25:17.311351 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.311326 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 23:25:17.313504 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.313317 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 23:25:17.314943 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.314925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:17.392185 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392185 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392404 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392633 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392858 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392858 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.392858 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.392708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffn8r\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-kube-api-access-ffn8r\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.493863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.493720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494215 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494313 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494474 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494444 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494549 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffn8r\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-kube-api-access-ffn8r\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.494981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.495332 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.494981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.495332 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.495121 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.496910 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.496754 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16e92d21-14c8-42c1-bf50-e5177c600bfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.497147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.497128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.497253 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.497233 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.498228 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.498099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.498228 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.498162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.498228 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.498205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.498507 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.498482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.498610 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.498582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.499669 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.499647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.499833 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.499816 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.500359 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.500344 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16e92d21-14c8-42c1-bf50-e5177c600bfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.500892 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.500870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16e92d21-14c8-42c1-bf50-e5177c600bfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.502613 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.502595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffn8r\" (UniqueName: \"kubernetes.io/projected/16e92d21-14c8-42c1-bf50-e5177c600bfa-kube-api-access-ffn8r\") pod \"prometheus-k8s-0\" (UID: \"16e92d21-14c8-42c1-bf50-e5177c600bfa\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.612019 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.611958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:25:17.764073 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:17.763981 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 23:25:17.766683 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:25:17.766657 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e92d21_14c8_42c1_bf50_e5177c600bfa.slice/crio-02cc434ae8e60b6b73bc82aa771e0eac3052e60447ac1da972c65912429ff202 WatchSource:0}: Error finding container 02cc434ae8e60b6b73bc82aa771e0eac3052e60447ac1da972c65912429ff202: Status 404 returned error can't find the container with id 02cc434ae8e60b6b73bc82aa771e0eac3052e60447ac1da972c65912429ff202 Jul 02 23:25:18.247103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:18.247066 2562 generic.go:356] "Generic (PLEG): container finished" podID="16e92d21-14c8-42c1-bf50-e5177c600bfa" containerID="41946a993584e3f4e23cfc4037f877f5c3c4024fba602fc92b8202f6a534eb11" exitCode=0 Jul 02 23:25:18.247551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:18.247160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerDied","Data":"41946a993584e3f4e23cfc4037f877f5c3c4024fba602fc92b8202f6a534eb11"} Jul 02 23:25:18.247551 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:18.247237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"02cc434ae8e60b6b73bc82aa771e0eac3052e60447ac1da972c65912429ff202"} Jul 02 23:25:18.434136 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:18.433252 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2e4e68e-0aaa-4afd-b121-314893af9730" path="/var/lib/kubelet/pods/d2e4e68e-0aaa-4afd-b121-314893af9730/volumes" Jul 02 23:25:19.254967 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.254929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"26e6f2a1c714e5555d7b4b097c0e239908eaafbd11a43ebd0bdf94dccb46fbab"} Jul 02 23:25:19.255323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.255039 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"de3301d2c395358117a669ca90f24e4f102b371afa7dd803517bb4df5b4dd6dd"} Jul 02 23:25:19.255323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.255073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"256f33eb4d075be9aec6e1f85bc8d477d334d33891443e1a00142e04bc4e1cf8"} Jul 02 23:25:19.255323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.255097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"fb8488ff229eb4bc60de65bf3d307f191ed3d25c60ccc13dc684a4ea1933c5d1"} Jul 02 23:25:19.255323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.255121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"06efbd3b4a62a9580414096746254c0933e60b6941d42b9b8ad66e8a06c3f1fd"} Jul 02 23:25:19.255323 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.255148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16e92d21-14c8-42c1-bf50-e5177c600bfa","Type":"ContainerStarted","Data":"03ad58ee1443ed09d129c3d89e8ddefaef582a89e27c0af886a6107a44e0c6e6"} Jul 02 23:25:19.285169 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:19.285107 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.268436051 podStartE2EDuration="2.285085304s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.649253ms" isStatefulPod=false podCreationTimestamp="2026-07-02 23:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:25:19.282971089 +0000 UTC m=+173.367572507" watchObservedRunningTime="2026-07-02 23:25:19.285085304 +0000 UTC m=+173.369686724" Jul 02 23:25:22.612411 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:25:22.612376 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:26:17.612295 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:26:17.612242 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:26:17.629044 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:26:17.628995 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:26:18.471405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:26:18.471379 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 23:27:26.303245 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:27:26.303208 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:28:28.543356 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.543276 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g"] Jul 02 23:28:28.546414 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.546398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.549035 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.548982 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:28:28.549153 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.549122 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:28:28.549374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.549353 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h86qq\"" Jul 02 23:28:28.549374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.549369 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:28:28.549530 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.549394 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:28:28.569788 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.569764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g"] Jul 02 23:28:28.611771 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.611735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.611950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.611776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.611950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.611804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls6w7\" (UniqueName: \"kubernetes.io/projected/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-kube-api-access-ls6w7\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.611950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.611878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.611950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.611935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712480 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712761 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712761 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls6w7\" (UniqueName: \"kubernetes.io/projected/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-kube-api-access-ls6w7\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.712966 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.712942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-manifests\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.714801 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.714777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-tmp\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.715202 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.715182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.715270 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.715257 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.719898 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.719873 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls6w7\" (UniqueName: \"kubernetes.io/projected/b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b-kube-api-access-ls6w7\") pod \"opendatahub-operator-controller-manager-6f45646c76-9dl8g\" (UID: \"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b\") " pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.856959 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.856876 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:28.991313 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.991279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g"] Jul 02 23:28:28.995254 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:28:28.995223 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35eb8ad_dac0_4ed4_a9d2_881ad7566d7b.slice/crio-08367ae1990bc8fd555f4e56cb233ce68c221a9f2678cb0ac5cab53fd9c815a9 WatchSource:0}: Error finding container 08367ae1990bc8fd555f4e56cb233ce68c221a9f2678cb0ac5cab53fd9c815a9: Status 404 returned error can't find the container with id 08367ae1990bc8fd555f4e56cb233ce68c221a9f2678cb0ac5cab53fd9c815a9 Jul 02 23:28:28.997579 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:28.997554 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:28:29.891459 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:29.891416 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" event={"ID":"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b","Type":"ContainerStarted","Data":"08367ae1990bc8fd555f4e56cb233ce68c221a9f2678cb0ac5cab53fd9c815a9"} Jul 02 23:28:32.903620 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:32.903582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" event={"ID":"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b","Type":"ContainerStarted","Data":"a55df696ba9f622d9e337095919cd2e9e15f2ce63485948b8729250578fb6f43"} Jul 02 23:28:33.908454 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:33.908425 2562 generic.go:356] "Generic (PLEG): container finished" podID="b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b" containerID="a55df696ba9f622d9e337095919cd2e9e15f2ce63485948b8729250578fb6f43" exitCode=0 Jul 02 23:28:33.908830 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:33.908486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" event={"ID":"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b","Type":"ContainerDied","Data":"a55df696ba9f622d9e337095919cd2e9e15f2ce63485948b8729250578fb6f43"} Jul 02 23:28:34.914153 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:34.914115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" event={"ID":"b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b","Type":"ContainerStarted","Data":"526b13f4c77d8f45dfa64117baa2028a434708404c7afce104c6b7995cbdb2c3"} Jul 02 23:28:34.914620 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:34.914224 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:34.935361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:34.935313 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" podStartSLOduration=2.520990204 podStartE2EDuration="6.935298916s" totalImagesPullingTime="3.327139028s" totalInitContainerRuntime="1.087169684s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:28:34.934267699 +0000 UTC m=+369.018869118" watchObservedRunningTime="2026-07-02 23:28:34.935298916 +0000 UTC m=+369.019900334" Jul 02 23:28:45.920742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:45.920706 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f45646c76-9dl8g" Jul 02 23:28:56.128646 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.128610 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx"] Jul 02 23:28:56.131579 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.131558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.135334 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135310 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 23:28:56.135483 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135424 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cppbc\"" Jul 02 23:28:56.135483 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135441 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:28:56.135606 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135572 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 23:28:56.135662 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135650 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:28:56.135902 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.135886 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 23:28:56.147768 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.147743 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx"] Jul 02 23:28:56.234626 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.234597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/59a24c20-0014-419c-a5dc-672abbb024da-manager-config\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.234807 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.234648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw2tc\" (UniqueName: \"kubernetes.io/projected/59a24c20-0014-419c-a5dc-672abbb024da-kube-api-access-kw2tc\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.234807 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.234704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.234807 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.234777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.312532 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.312500 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx"] Jul 02 23:28:56.315560 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.315545 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.318519 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.318487 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 23:28:56.318519 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.318492 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xmwj5\"" Jul 02 23:28:56.318712 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.318552 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:28:56.318712 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.318573 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:28:56.327047 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.326998 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx"] Jul 02 23:28:56.335997 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.335975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.336131 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.336028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/59a24c20-0014-419c-a5dc-672abbb024da-manager-config\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.336131 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.336078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw2tc\" (UniqueName: \"kubernetes.io/projected/59a24c20-0014-419c-a5dc-672abbb024da-kube-api-access-kw2tc\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.336131 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.336100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.336773 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.336714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/59a24c20-0014-419c-a5dc-672abbb024da-manager-config\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.338562 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.338542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-metrics-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.338654 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.338601 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59a24c20-0014-419c-a5dc-672abbb024da-cert\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.352200 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.352179 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw2tc\" (UniqueName: \"kubernetes.io/projected/59a24c20-0014-419c-a5dc-672abbb024da-kube-api-access-kw2tc\") pod \"lws-controller-manager-679bd8f54d-hv7kx\" (UID: \"59a24c20-0014-419c-a5dc-672abbb024da\") " pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.437105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.436995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb92l\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-kube-api-access-mb92l\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b7afa5-a99f-41ab-be30-31df7e653bcf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.437361 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.437330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.440236 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.440216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:28:56.538566 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.538534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.538730 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.538578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.538730 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.538610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.538730 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.538669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539030 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.538974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb92l\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-kube-api-access-mb92l\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539108 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539056 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539108 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b7afa5-a99f-41ab-be30-31df7e653bcf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539108 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539271 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539321 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539533 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539591 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539558 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539644 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.539949 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.539926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96b7afa5-a99f-41ab-be30-31df7e653bcf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.541418 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.541396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.541799 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.541780 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.549427 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.549401 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb92l\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-kube-api-access-mb92l\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.549522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.549427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96b7afa5-a99f-41ab-be30-31df7e653bcf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6787fvrx\" (UID: \"96b7afa5-a99f-41ab-be30-31df7e653bcf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.573170 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.573136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx"] Jul 02 23:28:56.577488 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:28:56.577462 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a24c20_0014_419c_a5dc_672abbb024da.slice/crio-336d6c173c5a9b286a97e65be8677d45e95e3c1797bc15c533fe0da3e68ced2f WatchSource:0}: Error finding container 336d6c173c5a9b286a97e65be8677d45e95e3c1797bc15c533fe0da3e68ced2f: Status 404 returned error can't find the container with id 336d6c173c5a9b286a97e65be8677d45e95e3c1797bc15c533fe0da3e68ced2f Jul 02 23:28:56.624960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.624931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:28:56.774507 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.774482 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx"] Jul 02 23:28:56.776646 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:28:56.776612 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b7afa5_a99f_41ab_be30_31df7e653bcf.slice/crio-cd5f762a0cdadcf1b13f6c65040457474b7a0218e7a78372463569f154f1420c WatchSource:0}: Error finding container cd5f762a0cdadcf1b13f6c65040457474b7a0218e7a78372463569f154f1420c: Status 404 returned error can't find the container with id cd5f762a0cdadcf1b13f6c65040457474b7a0218e7a78372463569f154f1420c Jul 02 23:28:56.993128 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.993091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" event={"ID":"59a24c20-0014-419c-a5dc-672abbb024da","Type":"ContainerStarted","Data":"336d6c173c5a9b286a97e65be8677d45e95e3c1797bc15c533fe0da3e68ced2f"} Jul 02 23:28:56.994277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:56.994254 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" event={"ID":"96b7afa5-a99f-41ab-be30-31df7e653bcf","Type":"ContainerStarted","Data":"cd5f762a0cdadcf1b13f6c65040457474b7a0218e7a78372463569f154f1420c"} Jul 02 23:28:59.431843 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.431803 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj"] Jul 02 23:28:59.438907 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.438881 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr"] Jul 02 23:28:59.439089 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.439071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.442120 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.442099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ssgbk\"" Jul 02 23:28:59.442992 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.442967 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 23:28:59.443513 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.443496 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.446777 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.446698 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 23:28:59.446895 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.446832 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hz4jv\"" Jul 02 23:28:59.460247 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.460226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj"] Jul 02 23:28:59.463152 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.462837 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr"] Jul 02 23:28:59.569126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.569088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a49512f8-964c-4134-be9d-f73f783a056d-cert\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.569297 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.569144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60eb6602-3774-40b3-a041-ed2ad78431f2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.569297 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.569172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5wm\" (UniqueName: \"kubernetes.io/projected/a49512f8-964c-4134-be9d-f73f783a056d-kube-api-access-cn5wm\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.569297 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.569212 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2ng9\" (UniqueName: \"kubernetes.io/projected/60eb6602-3774-40b3-a041-ed2ad78431f2-kube-api-access-d2ng9\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.670503 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.670455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a49512f8-964c-4134-be9d-f73f783a056d-cert\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.670678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.670518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60eb6602-3774-40b3-a041-ed2ad78431f2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.670678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.670544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5wm\" (UniqueName: \"kubernetes.io/projected/a49512f8-964c-4134-be9d-f73f783a056d-kube-api-access-cn5wm\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.670678 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.670572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2ng9\" (UniqueName: \"kubernetes.io/projected/60eb6602-3774-40b3-a041-ed2ad78431f2-kube-api-access-d2ng9\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.673501 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.673476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60eb6602-3774-40b3-a041-ed2ad78431f2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.673611 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.673521 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a49512f8-964c-4134-be9d-f73f783a056d-cert\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.679726 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.679694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5wm\" (UniqueName: \"kubernetes.io/projected/a49512f8-964c-4134-be9d-f73f783a056d-kube-api-access-cn5wm\") pod \"kserve-controller-manager-d6f4b65b8-hnrnj\" (UID: \"a49512f8-964c-4134-be9d-f73f783a056d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.679930 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.679909 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2ng9\" (UniqueName: \"kubernetes.io/projected/60eb6602-3774-40b3-a041-ed2ad78431f2-kube-api-access-d2ng9\") pod \"llmisvc-controller-manager-777c6bd7d5-5zpnr\" (UID: \"60eb6602-3774-40b3-a041-ed2ad78431f2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:28:59.752831 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.752793 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:28:59.760202 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:28:59.760177 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:29:00.166315 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:00.166279 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:29:00.166426 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:00.166362 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:29:00.166426 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:00.166393 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:29:00.336768 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:29:00.336731 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49512f8_964c_4134_be9d_f73f783a056d.slice/crio-5068a97fc31234e4d6e9bd9bb0c64caa479cb4fa88971e4569402e72574278d1 WatchSource:0}: Error finding container 5068a97fc31234e4d6e9bd9bb0c64caa479cb4fa88971e4569402e72574278d1: Status 404 returned error can't find the container with id 5068a97fc31234e4d6e9bd9bb0c64caa479cb4fa88971e4569402e72574278d1 Jul 02 23:29:00.339134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:00.339114 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj"] Jul 02 23:29:00.346130 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:00.346112 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr"] Jul 02 23:29:00.348350 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:29:00.348325 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60eb6602_3774_40b3_a041_ed2ad78431f2.slice/crio-4b6d95c35099834f6f9848d96939ecab210e621e0a8dde0f13b2405824e16e98 WatchSource:0}: Error finding container 4b6d95c35099834f6f9848d96939ecab210e621e0a8dde0f13b2405824e16e98: Status 404 returned error can't find the container with id 4b6d95c35099834f6f9848d96939ecab210e621e0a8dde0f13b2405824e16e98 Jul 02 23:29:01.014946 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.013697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" event={"ID":"59a24c20-0014-419c-a5dc-672abbb024da","Type":"ContainerStarted","Data":"a8ed4fb95d290d5c7e526e309611b6bccf2dbdd94d644c3f70b1aae32f9a610d"} Jul 02 23:29:01.014946 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.014575 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:29:01.017087 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.017055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" event={"ID":"60eb6602-3774-40b3-a041-ed2ad78431f2","Type":"ContainerStarted","Data":"4b6d95c35099834f6f9848d96939ecab210e621e0a8dde0f13b2405824e16e98"} Jul 02 23:29:01.019879 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.019832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" event={"ID":"96b7afa5-a99f-41ab-be30-31df7e653bcf","Type":"ContainerStarted","Data":"4248e26db4d3cbd840ccc3848ea1b722515c6c34defc4b94a104114766af9f06"} Jul 02 23:29:01.021336 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.021303 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" event={"ID":"a49512f8-964c-4134-be9d-f73f783a056d","Type":"ContainerStarted","Data":"5068a97fc31234e4d6e9bd9bb0c64caa479cb4fa88971e4569402e72574278d1"} Jul 02 23:29:01.035929 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.035884 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" podStartSLOduration=1.456902253 podStartE2EDuration="5.035862622s" totalImagesPullingTime="3.578960369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:29:01.034762134 +0000 UTC m=+395.119363555" watchObservedRunningTime="2026-07-02 23:29:01.035862622 +0000 UTC m=+395.120464042" Jul 02 23:29:01.625584 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.625541 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:29:01.631114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.631085 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:29:01.650953 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:01.650898 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" podStartSLOduration=2.2634472629999998 podStartE2EDuration="5.650880054s" totalImagesPullingTime="3.387432791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:29:01.055968073 +0000 UTC m=+395.140569526" watchObservedRunningTime="2026-07-02 23:29:01.650880054 +0000 UTC m=+395.735481472" Jul 02 23:29:02.027191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:02.027147 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:29:02.028450 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:02.028413 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6787fvrx" Jul 02 23:29:03.032803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:03.032769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" event={"ID":"a49512f8-964c-4134-be9d-f73f783a056d","Type":"ContainerStarted","Data":"99f527415a295e9efb5d926eff842a9b9c91ee946aa321dea2396c02a92c2b47"} Jul 02 23:29:03.033268 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:03.032869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:29:03.047675 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:03.047629 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" podStartSLOduration=1.721804007 podStartE2EDuration="4.047611046s" totalImagesPullingTime="2.325807039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:29:03.046841127 +0000 UTC m=+397.131442545" watchObservedRunningTime="2026-07-02 23:29:03.047611046 +0000 UTC m=+397.132212480" Jul 02 23:29:04.038178 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:04.038071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" event={"ID":"60eb6602-3774-40b3-a041-ed2ad78431f2","Type":"ContainerStarted","Data":"7720dcf80c98a7f787da606b78d976fd9232201ae42b33878abce0c542738832"} Jul 02 23:29:04.038580 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:04.038480 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:29:04.056173 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:04.056119 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" podStartSLOduration=1.681467609 podStartE2EDuration="5.056104597s" totalImagesPullingTime="3.374636988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:29:04.055195424 +0000 UTC m=+398.139796844" watchObservedRunningTime="2026-07-02 23:29:04.056104597 +0000 UTC m=+398.140706014" Jul 02 23:29:13.035104 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:13.035073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-679bd8f54d-hv7kx" Jul 02 23:29:34.043657 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:34.043625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hnrnj" Jul 02 23:29:36.048569 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:29:36.048537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5zpnr" Jul 02 23:30:25.027738 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.027700 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-87tdv"] Jul 02 23:30:25.029850 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.029833 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:25.032415 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.032391 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-m6xbx\"" Jul 02 23:30:25.032545 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.032432 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:30:25.033354 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.033339 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:30:25.039096 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.039068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-87tdv"] Jul 02 23:30:25.093522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.093478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lnv\" (UniqueName: \"kubernetes.io/projected/2fb7115d-336b-4633-8dfc-c818097bfcef-kube-api-access-74lnv\") pod \"authorino-operator-75cc8d5cb7-87tdv\" (UID: \"2fb7115d-336b-4633-8dfc-c818097bfcef\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:25.194804 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.194750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lnv\" (UniqueName: \"kubernetes.io/projected/2fb7115d-336b-4633-8dfc-c818097bfcef-kube-api-access-74lnv\") pod \"authorino-operator-75cc8d5cb7-87tdv\" (UID: \"2fb7115d-336b-4633-8dfc-c818097bfcef\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:25.204069 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.204038 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lnv\" (UniqueName: \"kubernetes.io/projected/2fb7115d-336b-4633-8dfc-c818097bfcef-kube-api-access-74lnv\") pod \"authorino-operator-75cc8d5cb7-87tdv\" (UID: \"2fb7115d-336b-4633-8dfc-c818097bfcef\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:25.340152 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.340053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:25.470485 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:25.470368 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-87tdv"] Jul 02 23:30:25.473289 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:30:25.473263 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb7115d_336b_4633_8dfc_c818097bfcef.slice/crio-9b8d844e396cda58390a2c8b7d5054a3b0915eda89abbb34b9bcc687eab1d20d WatchSource:0}: Error finding container 9b8d844e396cda58390a2c8b7d5054a3b0915eda89abbb34b9bcc687eab1d20d: Status 404 returned error can't find the container with id 9b8d844e396cda58390a2c8b7d5054a3b0915eda89abbb34b9bcc687eab1d20d Jul 02 23:30:26.364206 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:26.364166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" event={"ID":"2fb7115d-336b-4633-8dfc-c818097bfcef","Type":"ContainerStarted","Data":"9b8d844e396cda58390a2c8b7d5054a3b0915eda89abbb34b9bcc687eab1d20d"} Jul 02 23:30:28.373656 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:28.373613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" event={"ID":"2fb7115d-336b-4633-8dfc-c818097bfcef","Type":"ContainerStarted","Data":"5b82509ef1ef5c09b8febc9ef1f03585bd0df9e3d8557a98b372cd068c466a6a"} Jul 02 23:30:28.374069 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:28.373722 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:28.389668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:28.389610 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" podStartSLOduration=1.13182528 podStartE2EDuration="3.38959041s" totalImagesPullingTime="2.25776513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:30:28.388197354 +0000 UTC m=+482.472798773" watchObservedRunningTime="2026-07-02 23:30:28.38959041 +0000 UTC m=+482.474191828" Jul 02 23:30:39.380418 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:39.380390 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-87tdv" Jul 02 23:30:51.629089 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.629053 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq"] Jul 02 23:30:51.635264 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.635242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.638205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.638183 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ml5c8\"" Jul 02 23:30:51.644887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.644866 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq"] Jul 02 23:30:51.725919 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.725878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.726142 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.725945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8hfl\" (UniqueName: \"kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.826675 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.826638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8hfl\" (UniqueName: \"kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.826849 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.826709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.827106 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.827086 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.835987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.835969 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8hfl\" (UniqueName: \"kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:51.945586 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:51.945505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:52.073464 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.073435 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq"] Jul 02 23:30:52.076857 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:30:52.076830 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddce1c37_e24f_4d75_aa87_6f4ed90997d0.slice/crio-d4256168e7f7247b9a72ee4c70620ef03542dc70cf66da24768330f9167f2fa0 WatchSource:0}: Error finding container d4256168e7f7247b9a72ee4c70620ef03542dc70cf66da24768330f9167f2fa0: Status 404 returned error can't find the container with id d4256168e7f7247b9a72ee4c70620ef03542dc70cf66da24768330f9167f2fa0 Jul 02 23:30:52.433717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.433687 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq"] Jul 02 23:30:52.439684 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.439660 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq"] Jul 02 23:30:52.453725 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.453699 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:30:52.459082 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.459060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.467854 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.467830 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:30:52.633282 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.633243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5v8r\" (UniqueName: \"kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.633696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.633382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.734306 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.734262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5v8r\" (UniqueName: \"kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.734484 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.734366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.734858 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.734833 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.744045 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.743971 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5v8r\" (UniqueName: \"kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-kvvfh\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:52.771048 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:52.770993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:53.254454 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:53.254429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:30:53.281629 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:30:53.281582 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8788c7ee_c131_4f24_8ace_68629534794a.slice/crio-466dfb03aef6bbc86f5567c46b76e2f96b651dc80e327dac0d318234fbe218ad WatchSource:0}: Error finding container 466dfb03aef6bbc86f5567c46b76e2f96b651dc80e327dac0d318234fbe218ad: Status 404 returned error can't find the container with id 466dfb03aef6bbc86f5567c46b76e2f96b651dc80e327dac0d318234fbe218ad Jul 02 23:30:53.480047 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:53.479994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" event={"ID":"8788c7ee-c131-4f24-8ace-68629534794a","Type":"ContainerStarted","Data":"466dfb03aef6bbc86f5567c46b76e2f96b651dc80e327dac0d318234fbe218ad"} Jul 02 23:30:57.500978 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.500943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" event={"ID":"8788c7ee-c131-4f24-8ace-68629534794a","Type":"ContainerStarted","Data":"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44"} Jul 02 23:30:57.501507 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.501047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:30:57.502701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.502673 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" containerName="manager" containerID="cri-o://6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a" gracePeriod=2 Jul 02 23:30:57.519339 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:30:57.519309 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" is forbidden: User \"system:node:ip-10-0-130-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-160.ec2.internal' and this object" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:57.519469 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.519419 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" podStartSLOduration=2.311478422 podStartE2EDuration="5.519398193s" totalImagesPullingTime="3.207919771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:30:57.517165138 +0000 UTC m=+511.601766557" watchObservedRunningTime="2026-07-02 23:30:57.519398193 +0000 UTC m=+511.603999620" Jul 02 23:30:57.737531 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.737501 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:57.739774 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:30:57.739747 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" is forbidden: User \"system:node:ip-10-0-130-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-160.ec2.internal' and this object" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:57.882676 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.882595 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8hfl\" (UniqueName: \"kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl\") pod \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " Jul 02 23:30:57.882829 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.882682 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume\") pod \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\" (UID: \"ddce1c37-e24f-4d75-aa87-6f4ed90997d0\") " Jul 02 23:30:57.882986 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.882963 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ddce1c37-e24f-4d75-aa87-6f4ed90997d0" (UID: "ddce1c37-e24f-4d75-aa87-6f4ed90997d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:30:57.884923 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.884903 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl" (OuterVolumeSpecName: "kube-api-access-l8hfl") pod "ddce1c37-e24f-4d75-aa87-6f4ed90997d0" (UID: "ddce1c37-e24f-4d75-aa87-6f4ed90997d0"). InnerVolumeSpecName "kube-api-access-l8hfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:30:57.983732 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.983701 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8hfl\" (UniqueName: \"kubernetes.io/projected/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-kube-api-access-l8hfl\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:30:57.983732 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:57.983726 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ddce1c37-e24f-4d75-aa87-6f4ed90997d0-extensions-socket-volume\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:30:58.432423 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.432377 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" path="/var/lib/kubelet/pods/ddce1c37-e24f-4d75-aa87-6f4ed90997d0/volumes" Jul 02 23:30:58.507972 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.507941 2562 generic.go:356] "Generic (PLEG): container finished" podID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" containerID="6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a" exitCode=2 Jul 02 23:30:58.508407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.507983 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:58.508862 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.508049 2562 scope.go:117] "RemoveContainer" containerID="6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a" Jul 02 23:30:58.510581 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:30:58.510548 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" is forbidden: User \"system:node:ip-10-0-130-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-160.ec2.internal' and this object" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:58.512586 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:30:58.512565 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9h8xq\" is forbidden: User \"system:node:ip-10-0-130-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-160.ec2.internal' and this object" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9h8xq" Jul 02 23:30:58.517279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.517260 2562 scope.go:117] "RemoveContainer" containerID="6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a" Jul 02 23:30:58.517531 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:30:58.517514 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a\": container with ID starting with 6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a not found: ID does not exist" containerID="6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a" Jul 02 23:30:58.517580 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:30:58.517541 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a"} err="failed to get container status \"6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a\": rpc error: code = NotFound desc = could not find container \"6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a\": container with ID starting with 6794f8767ad959683184979fdc6ca73dbcb4ab636ece8810d091dd05e428938a not found: ID does not exist" Jul 02 23:31:08.510482 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:08.510454 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:31:21.004668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.004571 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:31:21.005197 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.004873 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" podUID="8788c7ee-c131-4f24-8ace-68629534794a" containerName="manager" containerID="cri-o://ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44" gracePeriod=10 Jul 02 23:31:21.264314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.264254 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:31:21.375579 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.375544 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5v8r\" (UniqueName: \"kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r\") pod \"8788c7ee-c131-4f24-8ace-68629534794a\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " Jul 02 23:31:21.375728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.375607 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume\") pod \"8788c7ee-c131-4f24-8ace-68629534794a\" (UID: \"8788c7ee-c131-4f24-8ace-68629534794a\") " Jul 02 23:31:21.376109 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.376087 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8788c7ee-c131-4f24-8ace-68629534794a" (UID: "8788c7ee-c131-4f24-8ace-68629534794a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:31:21.377824 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.377799 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r" (OuterVolumeSpecName: "kube-api-access-x5v8r") pod "8788c7ee-c131-4f24-8ace-68629534794a" (UID: "8788c7ee-c131-4f24-8ace-68629534794a"). InnerVolumeSpecName "kube-api-access-x5v8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:31:21.477095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.477055 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8788c7ee-c131-4f24-8ace-68629534794a-extensions-socket-volume\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.477095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.477085 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5v8r\" (UniqueName: \"kubernetes.io/projected/8788c7ee-c131-4f24-8ace-68629534794a-kube-api-access-x5v8r\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:31:21.605702 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.605603 2562 generic.go:356] "Generic (PLEG): container finished" podID="8788c7ee-c131-4f24-8ace-68629534794a" containerID="ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44" exitCode=0 Jul 02 23:31:21.605702 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.605646 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" Jul 02 23:31:21.605702 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.605691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" event={"ID":"8788c7ee-c131-4f24-8ace-68629534794a","Type":"ContainerDied","Data":"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44"} Jul 02 23:31:21.605957 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.605741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh" event={"ID":"8788c7ee-c131-4f24-8ace-68629534794a","Type":"ContainerDied","Data":"466dfb03aef6bbc86f5567c46b76e2f96b651dc80e327dac0d318234fbe218ad"} Jul 02 23:31:21.605957 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.605763 2562 scope.go:117] "RemoveContainer" containerID="ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44" Jul 02 23:31:21.614474 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.614456 2562 scope.go:117] "RemoveContainer" containerID="ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44" Jul 02 23:31:21.614732 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:31:21.614712 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44\": container with ID starting with ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44 not found: ID does not exist" containerID="ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44" Jul 02 23:31:21.614794 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.614741 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44"} err="failed to get container status \"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44\": rpc error: code = NotFound desc = could not find container \"ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44\": container with ID starting with ce5102006df8ac80f2662ba634de608d2ff9b7473cf843dd3b8a6baced6c5a44 not found: ID does not exist" Jul 02 23:31:21.640385 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.640353 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:31:21.655441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:21.655406 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-kvvfh"] Jul 02 23:31:22.431645 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:22.431611 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8788c7ee-c131-4f24-8ace-68629534794a" path="/var/lib/kubelet/pods/8788c7ee-c131-4f24-8ace-68629534794a/volumes" Jul 02 23:31:24.851475 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851434 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk"] Jul 02 23:31:24.851839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851761 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8788c7ee-c131-4f24-8ace-68629534794a" containerName="manager" Jul 02 23:31:24.851839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851773 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8788c7ee-c131-4f24-8ace-68629534794a" containerName="manager" Jul 02 23:31:24.851839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851793 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" containerName="manager" Jul 02 23:31:24.851839 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851798 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" containerName="manager" Jul 02 23:31:24.851962 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851844 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8788c7ee-c131-4f24-8ace-68629534794a" containerName="manager" Jul 02 23:31:24.851962 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.851851 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddce1c37-e24f-4d75-aa87-6f4ed90997d0" containerName="manager" Jul 02 23:31:24.856050 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.856029 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.858479 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.858458 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-d92fq\"" Jul 02 23:31:24.864835 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.864806 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk"] Jul 02 23:31:24.907351 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907699 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487b9ec5-8615-44ec-95cf-699ec5edbde5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907699 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg9qh\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-kube-api-access-jg9qh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907699 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:24.907699 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:24.907695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008163 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487b9ec5-8615-44ec-95cf-699ec5edbde5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg9qh\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-kube-api-access-jg9qh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.008803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.008769 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.009086 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.009032 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/487b9ec5-8615-44ec-95cf-699ec5edbde5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.010706 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.010684 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.010895 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.010879 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.016076 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.016051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.016271 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.016254 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg9qh\" (UniqueName: \"kubernetes.io/projected/487b9ec5-8615-44ec-95cf-699ec5edbde5-kube-api-access-jg9qh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2zgmk\" (UID: \"487b9ec5-8615-44ec-95cf-699ec5edbde5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.166836 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.166740 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:25.306933 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.306825 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk"] Jul 02 23:31:25.309670 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:31:25.309639 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487b9ec5_8615_44ec_95cf_699ec5edbde5.slice/crio-043d085bb0915965b1d7adc9aa13f78a596fd5fecf7ba5670f7ed0d62b2799fe WatchSource:0}: Error finding container 043d085bb0915965b1d7adc9aa13f78a596fd5fecf7ba5670f7ed0d62b2799fe: Status 404 returned error can't find the container with id 043d085bb0915965b1d7adc9aa13f78a596fd5fecf7ba5670f7ed0d62b2799fe Jul 02 23:31:25.311646 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.311612 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:31:25.311724 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.311680 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:31:25.311724 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.311708 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:31:25.626054 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.625993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" event={"ID":"487b9ec5-8615-44ec-95cf-699ec5edbde5","Type":"ContainerStarted","Data":"fc008aafe272085fa735d0af5754580574c268e74cf8517c4022a5b3f083e9d6"} Jul 02 23:31:25.626239 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.626078 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" event={"ID":"487b9ec5-8615-44ec-95cf-699ec5edbde5","Type":"ContainerStarted","Data":"043d085bb0915965b1d7adc9aa13f78a596fd5fecf7ba5670f7ed0d62b2799fe"} Jul 02 23:31:25.644027 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:25.643956 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" podStartSLOduration=1.6439387559999998 podStartE2EDuration="1.643938756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:25.642447293 +0000 UTC m=+539.727048708" watchObservedRunningTime="2026-07-02 23:31:25.643938756 +0000 UTC m=+539.728540174" Jul 02 23:31:26.167931 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:26.167894 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:26.173267 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:26.173240 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:26.630246 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:26.630211 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:26.631144 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:26.631127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:31:39.413956 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.413919 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:39.418247 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.418228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:39.420695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.420671 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gmx56\"" Jul 02 23:31:39.425132 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.425103 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:39.542823 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.542790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drfl2\" (UniqueName: \"kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2\") pod \"authorino-58cd955675-hhdkn\" (UID: \"c79f7179-735a-488e-b945-f075ac98fb8d\") " pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:39.643527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.643490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drfl2\" (UniqueName: \"kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2\") pod \"authorino-58cd955675-hhdkn\" (UID: \"c79f7179-735a-488e-b945-f075ac98fb8d\") " pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:39.651356 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.651331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drfl2\" (UniqueName: \"kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2\") pod \"authorino-58cd955675-hhdkn\" (UID: \"c79f7179-735a-488e-b945-f075ac98fb8d\") " pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:39.728743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.728706 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:39.861971 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:39.861940 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:39.866475 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:31:39.866440 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc79f7179_735a_488e_b945_f075ac98fb8d.slice/crio-e5734eecdbf2c662ff7b5de3f04a709ac537c52e88d8c78e2b93c4e43c4189ab WatchSource:0}: Error finding container e5734eecdbf2c662ff7b5de3f04a709ac537c52e88d8c78e2b93c4e43c4189ab: Status 404 returned error can't find the container with id e5734eecdbf2c662ff7b5de3f04a709ac537c52e88d8c78e2b93c4e43c4189ab Jul 02 23:31:40.687744 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:40.687713 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hhdkn" event={"ID":"c79f7179-735a-488e-b945-f075ac98fb8d","Type":"ContainerStarted","Data":"e5734eecdbf2c662ff7b5de3f04a709ac537c52e88d8c78e2b93c4e43c4189ab"} Jul 02 23:31:43.703346 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:43.703305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hhdkn" event={"ID":"c79f7179-735a-488e-b945-f075ac98fb8d","Type":"ContainerStarted","Data":"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2"} Jul 02 23:31:43.717358 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:43.717309 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hhdkn" podStartSLOduration=1.590763757 podStartE2EDuration="4.717295024s" totalImagesPullingTime="3.126531267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:31:43.716725092 +0000 UTC m=+557.801326510" watchObservedRunningTime="2026-07-02 23:31:43.717295024 +0000 UTC m=+557.801896441" Jul 02 23:31:44.344544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:44.344506 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:45.710671 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:45.710613 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hhdkn" podUID="c79f7179-735a-488e-b945-f075ac98fb8d" containerName="authorino" containerID="cri-o://056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2" gracePeriod=30 Jul 02 23:31:45.952911 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:45.952886 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:46.106724 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.106673 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drfl2\" (UniqueName: \"kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2\") pod \"c79f7179-735a-488e-b945-f075ac98fb8d\" (UID: \"c79f7179-735a-488e-b945-f075ac98fb8d\") " Jul 02 23:31:46.108875 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.108844 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2" (OuterVolumeSpecName: "kube-api-access-drfl2") pod "c79f7179-735a-488e-b945-f075ac98fb8d" (UID: "c79f7179-735a-488e-b945-f075ac98fb8d"). InnerVolumeSpecName "kube-api-access-drfl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:31:46.207417 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.207376 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drfl2\" (UniqueName: \"kubernetes.io/projected/c79f7179-735a-488e-b945-f075ac98fb8d-kube-api-access-drfl2\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:31:46.716335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.716227 2562 generic.go:356] "Generic (PLEG): container finished" podID="c79f7179-735a-488e-b945-f075ac98fb8d" containerID="056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2" exitCode=0 Jul 02 23:31:46.716335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.716276 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hhdkn" Jul 02 23:31:46.716866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.716269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hhdkn" event={"ID":"c79f7179-735a-488e-b945-f075ac98fb8d","Type":"ContainerDied","Data":"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2"} Jul 02 23:31:46.716866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.716385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hhdkn" event={"ID":"c79f7179-735a-488e-b945-f075ac98fb8d","Type":"ContainerDied","Data":"e5734eecdbf2c662ff7b5de3f04a709ac537c52e88d8c78e2b93c4e43c4189ab"} Jul 02 23:31:46.716866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.716410 2562 scope.go:117] "RemoveContainer" containerID="056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2" Jul 02 23:31:46.725618 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.725599 2562 scope.go:117] "RemoveContainer" containerID="056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2" Jul 02 23:31:46.725948 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:31:46.725929 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2\": container with ID starting with 056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2 not found: ID does not exist" containerID="056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2" Jul 02 23:31:46.726048 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.725961 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2"} err="failed to get container status \"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2\": rpc error: code = NotFound desc = could not find container \"056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2\": container with ID starting with 056d54fbdfbd752b70af8fcbb18278b7224b9fef6ed01ee4beffcd3be685f3d2 not found: ID does not exist" Jul 02 23:31:46.733137 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.733111 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:46.735893 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:46.735872 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hhdkn"] Jul 02 23:31:48.431785 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:31:48.431748 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c79f7179-735a-488e-b945-f075ac98fb8d" path="/var/lib/kubelet/pods/c79f7179-735a-488e-b945-f075ac98fb8d/volumes" Jul 02 23:32:53.426826 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.426745 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-28sgd"] Jul 02 23:32:53.427234 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.427095 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79f7179-735a-488e-b945-f075ac98fb8d" containerName="authorino" Jul 02 23:32:53.427234 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.427107 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79f7179-735a-488e-b945-f075ac98fb8d" containerName="authorino" Jul 02 23:32:53.427234 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.427154 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79f7179-735a-488e-b945-f075ac98fb8d" containerName="authorino" Jul 02 23:32:53.429747 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.429729 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.432240 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.432214 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lkhx2\"" Jul 02 23:32:53.432401 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.432385 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 02 23:32:53.440118 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.440093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-28sgd"] Jul 02 23:32:53.468646 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.468615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-plugins-config-volume\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.468809 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.468715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnjh\" (UniqueName: \"kubernetes.io/projected/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-kube-api-access-dgnjh\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.569271 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.569231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnjh\" (UniqueName: \"kubernetes.io/projected/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-kube-api-access-dgnjh\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.569271 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.569279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-plugins-config-volume\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.569919 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.569900 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-plugins-config-volume\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.576877 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.576846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnjh\" (UniqueName: \"kubernetes.io/projected/5664fc42-a2cf-4b95-84c3-dee1f3aa6289-kube-api-access-dgnjh\") pod \"payload-processing-58f86c7798-28sgd\" (UID: \"5664fc42-a2cf-4b95-84c3-dee1f3aa6289\") " pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.741078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.741043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:32:53.874310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.874283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-28sgd"] Jul 02 23:32:53.876965 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:32:53.876939 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5664fc42_a2cf_4b95_84c3_dee1f3aa6289.slice/crio-dbf67d38db5afe4f818dbf94e419d6ff1009a4d26c90477fa3816f9a8822323d WatchSource:0}: Error finding container dbf67d38db5afe4f818dbf94e419d6ff1009a4d26c90477fa3816f9a8822323d: Status 404 returned error can't find the container with id dbf67d38db5afe4f818dbf94e419d6ff1009a4d26c90477fa3816f9a8822323d Jul 02 23:32:53.998337 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:53.998246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" event={"ID":"5664fc42-a2cf-4b95-84c3-dee1f3aa6289","Type":"ContainerStarted","Data":"dbf67d38db5afe4f818dbf94e419d6ff1009a4d26c90477fa3816f9a8822323d"} Jul 02 23:32:56.009078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:56.009044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" event={"ID":"5664fc42-a2cf-4b95-84c3-dee1f3aa6289","Type":"ContainerStarted","Data":"63c7e78148fbd484cca5c7cab4eb8b05d3af2c45ae0a4021be1d6e41c9d26309"} Jul 02 23:32:56.009477 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:32:56.009129 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:33:01.035803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.035771 2562 generic.go:356] "Generic (PLEG): container finished" podID="487b9ec5-8615-44ec-95cf-699ec5edbde5" containerID="fc008aafe272085fa735d0af5754580574c268e74cf8517c4022a5b3f083e9d6" exitCode=0 Jul 02 23:33:01.036210 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.035840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" event={"ID":"487b9ec5-8615-44ec-95cf-699ec5edbde5","Type":"ContainerDied","Data":"fc008aafe272085fa735d0af5754580574c268e74cf8517c4022a5b3f083e9d6"} Jul 02 23:33:01.036251 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.036239 2562 scope.go:117] "RemoveContainer" containerID="fc008aafe272085fa735d0af5754580574c268e74cf8517c4022a5b3f083e9d6" Jul 02 23:33:01.037315 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.037288 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:33:01.037374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.037357 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:33:01.037411 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.037387 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:33:01.055244 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:01.055173 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" podStartSLOduration=6.627383786 podStartE2EDuration="8.055157987s" totalImagesPullingTime="1.427774201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:32:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:32:56.025780815 +0000 UTC m=+630.110382234" watchObservedRunningTime="2026-07-02 23:33:01.055157987 +0000 UTC m=+635.139759408" Jul 02 23:33:02.043571 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:02.043535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" event={"ID":"487b9ec5-8615-44ec-95cf-699ec5edbde5","Type":"ContainerStarted","Data":"b40242473acc30b2685b3fe8691501682679cfbfc52fc57beb3ac7d14aacb8bb"} Jul 02 23:33:02.167527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:02.167486 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:33:02.167970 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:02.167729 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2zgmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 02 23:33:02.167970 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:02.167790 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" podUID="487b9ec5-8615-44ec-95cf-699ec5edbde5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 02 23:33:03.168107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:03.168068 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2zgmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 02 23:33:03.168509 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:03.168124 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" podUID="487b9ec5-8615-44ec-95cf-699ec5edbde5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 02 23:33:04.167243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:04.167206 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2zgmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 02 23:33:04.167432 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:04.167258 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" podUID="487b9ec5-8615-44ec-95cf-699ec5edbde5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 02 23:33:05.167763 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:05.167730 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2zgmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 02 23:33:05.168154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:05.167786 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" podUID="487b9ec5-8615-44ec-95cf-699ec5edbde5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 02 23:33:06.171089 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:06.171058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:33:06.171473 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:06.171342 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:33:06.171995 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:06.171980 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2zgmk" Jul 02 23:33:07.015856 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:07.015825 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-28sgd" Jul 02 23:33:20.853667 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.853628 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc"] Jul 02 23:33:20.857556 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.857531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.860291 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.860265 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:33:20.861404 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.861384 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:33:20.861515 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.861405 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 23:33:20.861515 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.861384 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k5pld\"" Jul 02 23:33:20.867085 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.867057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc"] Jul 02 23:33:20.897616 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.897798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.897798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mnnr\" (UniqueName: \"kubernetes.io/projected/8bd092f0-05c5-42a5-837b-a5140dad7463-kube-api-access-4mnnr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.897798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.897798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bd092f0-05c5-42a5-837b-a5140dad7463-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.897962 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.897810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.998867 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.998828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999068 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.998923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999068 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.998979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999068 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mnnr\" (UniqueName: \"kubernetes.io/projected/8bd092f0-05c5-42a5-837b-a5140dad7463-kube-api-access-4mnnr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999068 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999273 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bd092f0-05c5-42a5-837b-a5140dad7463-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999332 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999282 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999332 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999299 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:20.999433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:20.999377 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:21.001502 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:21.001482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bd092f0-05c5-42a5-837b-a5140dad7463-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:21.001665 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:21.001649 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bd092f0-05c5-42a5-837b-a5140dad7463-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:21.006721 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:21.006697 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mnnr\" (UniqueName: \"kubernetes.io/projected/8bd092f0-05c5-42a5-837b-a5140dad7463-kube-api-access-4mnnr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc\" (UID: \"8bd092f0-05c5-42a5-837b-a5140dad7463\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:21.169776 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:21.169682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:21.304172 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:21.304142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc"] Jul 02 23:33:21.307983 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:33:21.307952 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd092f0_05c5_42a5_837b_a5140dad7463.slice/crio-000f25dd4e6098ae8c76ea0738d73c70b4093142a62e87fecb4496eb95fc2a52 WatchSource:0}: Error finding container 000f25dd4e6098ae8c76ea0738d73c70b4093142a62e87fecb4496eb95fc2a52: Status 404 returned error can't find the container with id 000f25dd4e6098ae8c76ea0738d73c70b4093142a62e87fecb4496eb95fc2a52 Jul 02 23:33:22.132237 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:22.132204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" event={"ID":"8bd092f0-05c5-42a5-837b-a5140dad7463","Type":"ContainerStarted","Data":"000f25dd4e6098ae8c76ea0738d73c70b4093142a62e87fecb4496eb95fc2a52"} Jul 02 23:33:23.137844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:23.137795 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" event={"ID":"8bd092f0-05c5-42a5-837b-a5140dad7463","Type":"ContainerStarted","Data":"664c49eb79d04e5373b96c51a5569814ad0e53cd6c9b772aa7fd4a987fcc8530"} Jul 02 23:33:23.154587 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:23.154532 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" podStartSLOduration=2.036205103 podStartE2EDuration="3.154516993s" totalImagesPullingTime="1.11831189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:33:23.153437529 +0000 UTC m=+657.238038960" watchObservedRunningTime="2026-07-02 23:33:23.154516993 +0000 UTC m=+657.239118437" Jul 02 23:33:31.169942 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:31.169908 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:31.170504 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:31.170057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:31.182559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:31.182538 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:32.185588 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.185559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc" Jul 02 23:33:32.763303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.763273 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc"] Jul 02 23:33:32.767035 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.766992 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.769461 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.769438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 02 23:33:32.776816 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.776790 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc"] Jul 02 23:33:32.905985 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.905949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.905985 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.905987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.906215 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.906037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcxm7\" (UniqueName: \"kubernetes.io/projected/56a66842-1c80-4232-8ef0-38e3f7014839-kube-api-access-xcxm7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.906215 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.906123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.906215 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.906165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:32.906215 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:32.906192 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a66842-1c80-4232-8ef0-38e3f7014839-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.006886 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.006848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.006886 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.006890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007184 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.006908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcxm7\" (UniqueName: \"kubernetes.io/projected/56a66842-1c80-4232-8ef0-38e3f7014839-kube-api-access-xcxm7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007184 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.006947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007184 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.006978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007184 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.007026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a66842-1c80-4232-8ef0-38e3f7014839-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.007356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.007401 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.007492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.007431 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.009397 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.009374 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56a66842-1c80-4232-8ef0-38e3f7014839-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.009538 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.009522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a66842-1c80-4232-8ef0-38e3f7014839-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.014283 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.014225 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcxm7\" (UniqueName: \"kubernetes.io/projected/56a66842-1c80-4232-8ef0-38e3f7014839-kube-api-access-xcxm7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gmtnc\" (UID: \"56a66842-1c80-4232-8ef0-38e3f7014839\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.080674 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.080631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:33.213370 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.213343 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc"] Jul 02 23:33:33.215663 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:33:33.215619 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a66842_1c80_4232_8ef0_38e3f7014839.slice/crio-87de89b5d697ef58dbb79443d80c4e8467fd3a920abce02bdc957682c1515827 WatchSource:0}: Error finding container 87de89b5d697ef58dbb79443d80c4e8467fd3a920abce02bdc957682c1515827: Status 404 returned error can't find the container with id 87de89b5d697ef58dbb79443d80c4e8467fd3a920abce02bdc957682c1515827 Jul 02 23:33:33.217317 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:33.217293 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:33:34.184510 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:34.184477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" event={"ID":"56a66842-1c80-4232-8ef0-38e3f7014839","Type":"ContainerStarted","Data":"ba8cfba3ad51d6d119b3ee4f131c0c336305e94cef82bc640a40372e96d37a25"} Jul 02 23:33:34.184677 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:34.184523 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" event={"ID":"56a66842-1c80-4232-8ef0-38e3f7014839","Type":"ContainerStarted","Data":"87de89b5d697ef58dbb79443d80c4e8467fd3a920abce02bdc957682c1515827"} Jul 02 23:33:34.204214 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:34.204163 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" podStartSLOduration=2.032215314 podStartE2EDuration="2.204144838s" totalImagesPullingTime="171.929524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:33:34.203041613 +0000 UTC m=+668.287643035" watchObservedRunningTime="2026-07-02 23:33:34.204144838 +0000 UTC m=+668.288746256" Jul 02 23:33:43.081846 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:43.081795 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:43.082316 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:43.081870 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:43.094682 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:43.094655 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:33:43.233753 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:33:43.233722 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gmtnc" Jul 02 23:35:58.463299 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.463221 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-99ffcd69c-llw4h"] Jul 02 23:35:58.467374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.467351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.470768 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.470738 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 23:35:58.470901 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.470807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gmx56\"" Jul 02 23:35:58.478441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.478410 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99ffcd69c-llw4h"] Jul 02 23:35:58.583278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.583239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/47c106ae-ac92-4b1e-953f-3d9e6ef08071-tls-cert\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.583460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.583287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhrzm\" (UniqueName: \"kubernetes.io/projected/47c106ae-ac92-4b1e-953f-3d9e6ef08071-kube-api-access-lhrzm\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.683910 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.683870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/47c106ae-ac92-4b1e-953f-3d9e6ef08071-tls-cert\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.684109 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.683921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhrzm\" (UniqueName: \"kubernetes.io/projected/47c106ae-ac92-4b1e-953f-3d9e6ef08071-kube-api-access-lhrzm\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.686534 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.686508 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/47c106ae-ac92-4b1e-953f-3d9e6ef08071-tls-cert\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.691300 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.691274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhrzm\" (UniqueName: \"kubernetes.io/projected/47c106ae-ac92-4b1e-953f-3d9e6ef08071-kube-api-access-lhrzm\") pod \"authorino-99ffcd69c-llw4h\" (UID: \"47c106ae-ac92-4b1e-953f-3d9e6ef08071\") " pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.777144 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.777087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99ffcd69c-llw4h" Jul 02 23:35:58.911507 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:58.911480 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99ffcd69c-llw4h"] Jul 02 23:35:58.913955 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:35:58.913930 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47c106ae_ac92_4b1e_953f_3d9e6ef08071.slice/crio-db5d3b386b7758fdcacae47790df437c635297c9f086a1ef0c41102d0325a72b WatchSource:0}: Error finding container db5d3b386b7758fdcacae47790df437c635297c9f086a1ef0c41102d0325a72b: Status 404 returned error can't find the container with id db5d3b386b7758fdcacae47790df437c635297c9f086a1ef0c41102d0325a72b Jul 02 23:35:59.800243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:59.799449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99ffcd69c-llw4h" event={"ID":"47c106ae-ac92-4b1e-953f-3d9e6ef08071","Type":"ContainerStarted","Data":"49f909d1ad33295c9cdafaa790102870bdd0850618457443ae2214ad523245c3"} Jul 02 23:35:59.800243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:59.799503 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99ffcd69c-llw4h" event={"ID":"47c106ae-ac92-4b1e-953f-3d9e6ef08071","Type":"ContainerStarted","Data":"db5d3b386b7758fdcacae47790df437c635297c9f086a1ef0c41102d0325a72b"} Jul 02 23:35:59.818594 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:35:59.818533 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-99ffcd69c-llw4h" podStartSLOduration=1.183372175 podStartE2EDuration="1.818509737s" totalImagesPullingTime="635.137562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:35:59.816799832 +0000 UTC m=+813.901401263" watchObservedRunningTime="2026-07-02 23:35:59.818509737 +0000 UTC m=+813.903111155" Jul 02 23:49:18.883260 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.883170 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:18.886683 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.886665 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.889255 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.889234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-90a204bb-openshift-default-dockercfg-s44v7\"" Jul 02 23:49:18.903527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.903504 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:18.936161 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936130 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936161 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936399 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936399 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936233 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936399 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936399 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r88xk\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936606 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936606 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:18.936606 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:18.936504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037136 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037335 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r88xk\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037617 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037741 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.037920 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037791 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.038032 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.037983 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.038078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.038045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.039933 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.039909 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.040078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.039992 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.044979 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.044956 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.045925 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.045897 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r88xk\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk\") pod \"e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.197062 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.196977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:19.333078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.333050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:19.335133 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:49:19.335100 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d775e3_9833_4b16_9aae_1dba73f9850e.slice/crio-46abe1b857c095067b0436ae0a37e83ca6c88cea8b73241ae351ebf9e219dd72 WatchSource:0}: Error finding container 46abe1b857c095067b0436ae0a37e83ca6c88cea8b73241ae351ebf9e219dd72: Status 404 returned error can't find the container with id 46abe1b857c095067b0436ae0a37e83ca6c88cea8b73241ae351ebf9e219dd72 Jul 02 23:49:19.337428 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.337395 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:19.337509 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.337463 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:19.337509 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:19.337496 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:20.252494 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:20.252457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" event={"ID":"52d775e3-9833-4b16-9aae-1dba73f9850e","Type":"ContainerStarted","Data":"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5"} Jul 02 23:49:20.252941 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:20.252507 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" event={"ID":"52d775e3-9833-4b16-9aae-1dba73f9850e","Type":"ContainerStarted","Data":"46abe1b857c095067b0436ae0a37e83ca6c88cea8b73241ae351ebf9e219dd72"} Jul 02 23:49:20.301861 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:20.301810 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" podStartSLOduration=2.301793335 podStartE2EDuration="2.301793335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:49:20.300562381 +0000 UTC m=+1614.385163800" watchObservedRunningTime="2026-07-02 23:49:20.301793335 +0000 UTC m=+1614.386394752" Jul 02 23:49:21.197417 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:21.197379 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:21.202766 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:21.202740 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:21.257645 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:21.257611 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:21.258584 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:21.258562 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:25.429527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:25.429479 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:25.430163 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:25.429701 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" podUID="52d775e3-9833-4b16-9aae-1dba73f9850e" containerName="istio-proxy" containerID="cri-o://a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5" gracePeriod=30 Jul 02 23:49:31.274252 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.274228 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:31.304103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.303990 2562 generic.go:356] "Generic (PLEG): container finished" podID="52d775e3-9833-4b16-9aae-1dba73f9850e" containerID="a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5" exitCode=0 Jul 02 23:49:31.304103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.304044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" event={"ID":"52d775e3-9833-4b16-9aae-1dba73f9850e","Type":"ContainerDied","Data":"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5"} Jul 02 23:49:31.304103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.304066 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" Jul 02 23:49:31.304314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.304110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x" event={"ID":"52d775e3-9833-4b16-9aae-1dba73f9850e","Type":"ContainerDied","Data":"46abe1b857c095067b0436ae0a37e83ca6c88cea8b73241ae351ebf9e219dd72"} Jul 02 23:49:31.304314 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.304141 2562 scope.go:117] "RemoveContainer" containerID="a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5" Jul 02 23:49:31.313544 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.313452 2562 scope.go:117] "RemoveContainer" containerID="a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5" Jul 02 23:49:31.315098 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:49:31.315070 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5\": container with ID starting with a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5 not found: ID does not exist" containerID="a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5" Jul 02 23:49:31.315198 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.315110 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5"} err="failed to get container status \"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5\": rpc error: code = NotFound desc = could not find container \"a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5\": container with ID starting with a6084fd49ca524c534e8b8bfa1431ed936a22fbccf5ad1c56b4fa1579d5075d5 not found: ID does not exist" Jul 02 23:49:31.354615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354590 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354622 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354659 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r88xk\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354716 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354879 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354751 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354879 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354781 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.354879 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354814 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.355041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354898 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:31.355041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.354954 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.355166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355132 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:31.355166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355154 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:31.355266 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355226 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert\") pod \"52d775e3-9833-4b16-9aae-1dba73f9850e\" (UID: \"52d775e3-9833-4b16-9aae-1dba73f9850e\") " Jul 02 23:49:31.355615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355562 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data" (OuterVolumeSpecName: "istio-data") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:31.355615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355573 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.355615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355590 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:49:31.355615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355610 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.355882 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.355624 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.357238 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.357206 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:31.357355 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.357264 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token" (OuterVolumeSpecName: "istio-token") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:31.357509 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.357493 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk" (OuterVolumeSpecName: "kube-api-access-r88xk") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "kube-api-access-r88xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:31.357509 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.357500 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52d775e3-9833-4b16-9aae-1dba73f9850e" (UID: "52d775e3-9833-4b16-9aae-1dba73f9850e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:49:31.456421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456386 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52d775e3-9833-4b16-9aae-1dba73f9850e-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.456421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456415 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.456421 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456425 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r88xk\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-kube-api-access-r88xk\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.456650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456435 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.456650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456445 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.456650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.456452 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52d775e3-9833-4b16-9aae-1dba73f9850e-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:31.639756 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.639722 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:31.654668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.654636 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-90a204bb-openshift-default-77b59c846c-p8h2x"] Jul 02 23:49:31.753631 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.753602 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:31.753946 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.753934 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d775e3-9833-4b16-9aae-1dba73f9850e" containerName="istio-proxy" Jul 02 23:49:31.753994 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.753949 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d775e3-9833-4b16-9aae-1dba73f9850e" containerName="istio-proxy" Jul 02 23:49:31.754050 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.754033 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d775e3-9833-4b16-9aae-1dba73f9850e" containerName="istio-proxy" Jul 02 23:49:31.756986 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.756966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.759857 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.759834 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-914dee6c-gw-openshift-default-dockercfg-877sg\"" Jul 02 23:49:31.770166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.770142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:31.859200 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859200 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxvth\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859465 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859465 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859288 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859587 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859639 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859709 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859709 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.859800 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.859731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960583 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960583 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxvth\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960583 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.960844 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.960940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.961162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961336 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.961224 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961336 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.961295 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.961668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.961540 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.963198 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.963169 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.963394 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.963377 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.967936 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.967915 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:31.968125 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:31.968109 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxvth\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth\") pod \"e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:32.068904 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.068877 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:32.201727 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.201702 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:32.203674 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:49:32.203650 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70ce419_666d_485e_a60e_eb10cff4536e.slice/crio-60acf5736a394d47402b9bf6d731234a81df91e94286a3ecb4645586d3606df4 WatchSource:0}: Error finding container 60acf5736a394d47402b9bf6d731234a81df91e94286a3ecb4645586d3606df4: Status 404 returned error can't find the container with id 60acf5736a394d47402b9bf6d731234a81df91e94286a3ecb4645586d3606df4 Jul 02 23:49:32.206015 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.205970 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:32.206087 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.206062 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:32.206120 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.206092 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:32.311783 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.311745 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" event={"ID":"c70ce419-666d-485e-a60e-eb10cff4536e","Type":"ContainerStarted","Data":"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b"} Jul 02 23:49:32.312276 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.311808 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" event={"ID":"c70ce419-666d-485e-a60e-eb10cff4536e","Type":"ContainerStarted","Data":"60acf5736a394d47402b9bf6d731234a81df91e94286a3ecb4645586d3606df4"} Jul 02 23:49:32.335717 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.335664 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" podStartSLOduration=1.335645818 podStartE2EDuration="1.335645818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:49:32.33394361 +0000 UTC m=+1626.418545029" watchObservedRunningTime="2026-07-02 23:49:32.335645818 +0000 UTC m=+1626.420247235" Jul 02 23:49:32.433362 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:32.433329 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d775e3-9833-4b16-9aae-1dba73f9850e" path="/var/lib/kubelet/pods/52d775e3-9833-4b16-9aae-1dba73f9850e/volumes" Jul 02 23:49:33.069893 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:33.069860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:33.074973 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:33.074946 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:33.318420 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:33.318386 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:33.319526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:33.319505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:37.818642 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:37.818600 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:37.819090 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:37.818805 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" podUID="c70ce419-666d-485e-a60e-eb10cff4536e" containerName="istio-proxy" containerID="cri-o://9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b" gracePeriod=30 Jul 02 23:49:43.064560 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.064537 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:43.159701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159604 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxvth\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.159701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159664 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.159701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159682 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.159701 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159704 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159729 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159751 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159786 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159815 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.159845 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket\") pod \"c70ce419-666d-485e-a60e-eb10cff4536e\" (UID: \"c70ce419-666d-485e-a60e-eb10cff4536e\") " Jul 02 23:49:43.160343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.160127 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:43.160343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.160142 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:49:43.160343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.160274 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:43.160343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.160327 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:43.160711 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.160339 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:43.162072 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.162040 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:49:43.162183 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.162088 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:43.162183 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.162102 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth" (OuterVolumeSpecName: "kube-api-access-kxvth") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "kube-api-access-kxvth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:43.162502 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.162478 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c70ce419-666d-485e-a60e-eb10cff4536e" (UID: "c70ce419-666d-485e-a60e-eb10cff4536e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:43.261353 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261311 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261353 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261347 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261353 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261357 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxvth\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-kube-api-access-kxvth\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261353 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261367 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ce419-666d-485e-a60e-eb10cff4536e-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261614 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261375 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ce419-666d-485e-a60e-eb10cff4536e-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261614 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261382 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ce419-666d-485e-a60e-eb10cff4536e-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261614 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261390 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261614 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261397 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.261614 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.261404 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ce419-666d-485e-a60e-eb10cff4536e-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:49:43.376526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.371711 2562 generic.go:356] "Generic (PLEG): container finished" podID="c70ce419-666d-485e-a60e-eb10cff4536e" containerID="9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b" exitCode=0 Jul 02 23:49:43.376526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.371868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" event={"ID":"c70ce419-666d-485e-a60e-eb10cff4536e","Type":"ContainerDied","Data":"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b"} Jul 02 23:49:43.376526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.371951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" event={"ID":"c70ce419-666d-485e-a60e-eb10cff4536e","Type":"ContainerDied","Data":"60acf5736a394d47402b9bf6d731234a81df91e94286a3ecb4645586d3606df4"} Jul 02 23:49:43.376526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.371992 2562 scope.go:117] "RemoveContainer" containerID="9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b" Jul 02 23:49:43.376526 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.372237 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf" Jul 02 23:49:43.386197 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.386178 2562 scope.go:117] "RemoveContainer" containerID="9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b" Jul 02 23:49:43.386436 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:49:43.386417 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b\": container with ID starting with 9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b not found: ID does not exist" containerID="9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b" Jul 02 23:49:43.386510 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.386444 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b"} err="failed to get container status \"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b\": rpc error: code = NotFound desc = could not find container \"9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b\": container with ID starting with 9302daff4257f8e298c494403d07e59404b5c98dc52402ef08d9247261a55e2b not found: ID does not exist" Jul 02 23:49:43.397422 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.397399 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:43.403113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.403093 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-914dee6c-gw-openshift-default-7df55d987f-x8pxf"] Jul 02 23:49:43.694738 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.694705 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:49:43.695257 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.695239 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c70ce419-666d-485e-a60e-eb10cff4536e" containerName="istio-proxy" Jul 02 23:49:43.695325 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.695262 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70ce419-666d-485e-a60e-eb10cff4536e" containerName="istio-proxy" Jul 02 23:49:43.695368 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.695356 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c70ce419-666d-485e-a60e-eb10cff4536e" containerName="istio-proxy" Jul 02 23:49:43.699721 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.699697 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.702108 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.702086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-caf3ad89-openshift-default-dockercfg-7mvrw\"" Jul 02 23:49:43.710542 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.710517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:49:43.766951 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.766919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.766963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.766992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767122 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767325 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.767325 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.767164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fshd4\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fshd4\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868974 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868813 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868974 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.868974 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868899 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.869113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.868994 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.869399 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.869383 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.871064 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.871045 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.871244 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.871227 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.875934 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.875904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:43.876105 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:43.876089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fshd4\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4\") pod \"e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:44.012222 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.012196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:44.151179 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.151150 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:49:44.153217 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:49:44.153176 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a738ab_5bcb_443d_b956_afe0a18dc793.slice/crio-dabb2b9995d74df7bdff6a345aa039ece43c98f63a87c1bc91cc0bc732bd7d69 WatchSource:0}: Error finding container dabb2b9995d74df7bdff6a345aa039ece43c98f63a87c1bc91cc0bc732bd7d69: Status 404 returned error can't find the container with id dabb2b9995d74df7bdff6a345aa039ece43c98f63a87c1bc91cc0bc732bd7d69 Jul 02 23:49:44.155414 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.155379 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:44.155511 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.155448 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:44.155511 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.155480 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:49:44.380156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.380075 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" event={"ID":"b4a738ab-5bcb-443d-b956-afe0a18dc793","Type":"ContainerStarted","Data":"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae"} Jul 02 23:49:44.380156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.380138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" event={"ID":"b4a738ab-5bcb-443d-b956-afe0a18dc793","Type":"ContainerStarted","Data":"dabb2b9995d74df7bdff6a345aa039ece43c98f63a87c1bc91cc0bc732bd7d69"} Jul 02 23:49:44.397723 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.397673 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" podStartSLOduration=1.397657198 podStartE2EDuration="1.397657198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:49:44.396445288 +0000 UTC m=+1638.481046705" watchObservedRunningTime="2026-07-02 23:49:44.397657198 +0000 UTC m=+1638.482258674" Jul 02 23:49:44.432644 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:44.432611 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c70ce419-666d-485e-a60e-eb10cff4536e" path="/var/lib/kubelet/pods/c70ce419-666d-485e-a60e-eb10cff4536e/volumes" Jul 02 23:49:45.013303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:45.013266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:45.018415 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:45.018389 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:45.385021 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:45.384928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:45.385766 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:45.385750 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:49:56.386339 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:56.386303 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:49:56.386817 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:49:56.386615 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerName="istio-proxy" containerID="cri-o://1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae" gracePeriod=30 Jul 02 23:50:00.387091 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:00.387046 2562 patch_prober.go:28] interesting pod/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 02 23:50:00.387456 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:00.387113 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 23:50:01.640996 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.640973 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:50:01.722940 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.722912 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723125 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.722971 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723125 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.722995 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fshd4\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723133 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723208 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723178 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723211 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723245 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723268 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723260 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:01.723310 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data\") pod \"b4a738ab-5bcb-443d-b956-afe0a18dc793\" (UID: \"b4a738ab-5bcb-443d-b956-afe0a18dc793\") " Jul 02 23:50:01.723555 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723397 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:01.723555 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723470 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:01.723668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723626 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.723668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723638 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:01.723668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723646 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.723668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723663 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.724056 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.723717 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:50:01.725419 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.725390 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4" (OuterVolumeSpecName: "kube-api-access-fshd4") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "kube-api-access-fshd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:01.725532 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.725428 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:50:01.725532 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.725448 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:01.725903 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.725878 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4a738ab-5bcb-443d-b956-afe0a18dc793" (UID: "b4a738ab-5bcb-443d-b956-afe0a18dc793"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:01.824796 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824753 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.824796 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824785 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.824796 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824799 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fshd4\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-kube-api-access-fshd4\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.825078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824813 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.825078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824824 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a738ab-5bcb-443d-b956-afe0a18dc793-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:01.825078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:01.824836 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a738ab-5bcb-443d-b956-afe0a18dc793-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:02.308225 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.308192 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:02.308547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.308534 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerName="istio-proxy" Jul 02 23:50:02.308591 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.308551 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerName="istio-proxy" Jul 02 23:50:02.308624 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.308619 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerName="istio-proxy" Jul 02 23:50:02.311718 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.311700 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.314244 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.314219 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8b3bd569-openshift-default-dockercfg-s5r9l\"" Jul 02 23:50:02.323628 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.323605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:02.429562 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429562 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x759k\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.429863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.430079 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.429915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.464808 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.464774 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4a738ab-5bcb-443d-b956-afe0a18dc793" containerID="1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae" exitCode=0 Jul 02 23:50:02.464971 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.464825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" event={"ID":"b4a738ab-5bcb-443d-b956-afe0a18dc793","Type":"ContainerDied","Data":"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae"} Jul 02 23:50:02.464971 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.464890 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" event={"ID":"b4a738ab-5bcb-443d-b956-afe0a18dc793","Type":"ContainerDied","Data":"dabb2b9995d74df7bdff6a345aa039ece43c98f63a87c1bc91cc0bc732bd7d69"} Jul 02 23:50:02.464971 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.464826 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg" Jul 02 23:50:02.464971 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.464924 2562 scope.go:117] "RemoveContainer" containerID="1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae" Jul 02 23:50:02.475477 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.475457 2562 scope.go:117] "RemoveContainer" containerID="1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae" Jul 02 23:50:02.475769 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:50:02.475751 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae\": container with ID starting with 1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae not found: ID does not exist" containerID="1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae" Jul 02 23:50:02.475811 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.475779 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae"} err="failed to get container status \"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae\": rpc error: code = NotFound desc = could not find container \"1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae\": container with ID starting with 1186f6e9d6ce14e4beefa61a39a6fc29253456b34caa23a7fd85b3f28f8f5dae not found: ID does not exist" Jul 02 23:50:02.483089 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.483062 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:50:02.487048 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.487023 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-caf3ad89-openshift-default-76cd4cd5-k9fhg"] Jul 02 23:50:02.530788 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.530750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.530788 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.530795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.530996 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.530811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.530996 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.530940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.530996 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.530985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x759k\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531267 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531267 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531267 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531241 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531412 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531305 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531573 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531666 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.531866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.531849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.533431 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.533400 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.533618 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.533602 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.538805 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.538783 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.538936 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.538916 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x759k\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k\") pod \"e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.623780 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.623694 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:02.765383 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.765190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:02.768178 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:50:02.768149 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1e69c0_fb60_4141_bbe6_d8370e1e505c.slice/crio-009536e83532c6512ea76d5eb0ec2401dac551153a944f4a08cf2cb1c8fe9979 WatchSource:0}: Error finding container 009536e83532c6512ea76d5eb0ec2401dac551153a944f4a08cf2cb1c8fe9979: Status 404 returned error can't find the container with id 009536e83532c6512ea76d5eb0ec2401dac551153a944f4a08cf2cb1c8fe9979 Jul 02 23:50:02.770303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.770266 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:50:02.770391 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.770339 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:50:02.770391 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:02.770368 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:50:03.470859 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.470824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" event={"ID":"4a1e69c0-fb60-4141-bbe6-d8370e1e505c","Type":"ContainerStarted","Data":"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42"} Jul 02 23:50:03.471079 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.470887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" event={"ID":"4a1e69c0-fb60-4141-bbe6-d8370e1e505c","Type":"ContainerStarted","Data":"009536e83532c6512ea76d5eb0ec2401dac551153a944f4a08cf2cb1c8fe9979"} Jul 02 23:50:03.491374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.491318 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" podStartSLOduration=1.491298327 podStartE2EDuration="1.491298327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:50:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:50:03.489903363 +0000 UTC m=+1657.574504783" watchObservedRunningTime="2026-07-02 23:50:03.491298327 +0000 UTC m=+1657.575899746" Jul 02 23:50:03.623893 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.623805 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:03.624982 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.624951 2562 patch_prober.go:28] interesting pod/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 02 23:50:03.625111 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:03.625047 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 02 23:50:04.431915 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:04.431881 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a738ab-5bcb-443d-b956-afe0a18dc793" path="/var/lib/kubelet/pods/b4a738ab-5bcb-443d-b956-afe0a18dc793/volumes" Jul 02 23:50:04.628626 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:04.628591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:05.481967 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:05.481921 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:05.483193 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:05.483164 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:08.163160 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:08.163128 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:08.495610 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:08.495555 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerName="istio-proxy" containerID="cri-o://418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42" gracePeriod=30 Jul 02 23:50:13.755718 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.755657 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:13.830698 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.830852 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830719 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.830852 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830776 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.830852 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830811 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.830852 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830843 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.831041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830898 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.831041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830938 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x759k\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.831041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830962 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.831041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.830988 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:13.831041 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831021 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs\") pod \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\" (UID: \"4a1e69c0-fb60-4141-bbe6-d8370e1e505c\") " Jul 02 23:50:13.831287 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831091 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:13.831351 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831313 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.831351 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831321 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:13.831460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831357 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:13.831790 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.831703 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:50:13.833160 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.833128 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:13.833278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.833183 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:13.833278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.833189 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:50:13.833278 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.833190 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k" (OuterVolumeSpecName: "kube-api-access-x759k") pod "4a1e69c0-fb60-4141-bbe6-d8370e1e505c" (UID: "4a1e69c0-fb60-4141-bbe6-d8370e1e505c"). InnerVolumeSpecName "kube-api-access-x759k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:13.931784 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931747 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.931784 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931777 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x759k\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-kube-api-access-x759k\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.931784 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931788 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.932037 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931796 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.932037 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931803 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.932037 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931810 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.932037 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931818 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:13.932037 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:13.931825 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a1e69c0-fb60-4141-bbe6-d8370e1e505c-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:50:14.523457 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.523424 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerID="418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42" exitCode=0 Jul 02 23:50:14.523617 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.523466 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" Jul 02 23:50:14.523617 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.523484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" event={"ID":"4a1e69c0-fb60-4141-bbe6-d8370e1e505c","Type":"ContainerDied","Data":"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42"} Jul 02 23:50:14.523617 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.523532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5" event={"ID":"4a1e69c0-fb60-4141-bbe6-d8370e1e505c","Type":"ContainerDied","Data":"009536e83532c6512ea76d5eb0ec2401dac551153a944f4a08cf2cb1c8fe9979"} Jul 02 23:50:14.523617 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.523553 2562 scope.go:117] "RemoveContainer" containerID="418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42" Jul 02 23:50:14.531989 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.531969 2562 scope.go:117] "RemoveContainer" containerID="418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42" Jul 02 23:50:14.532268 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:50:14.532248 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42\": container with ID starting with 418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42 not found: ID does not exist" containerID="418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42" Jul 02 23:50:14.532338 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.532276 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42"} err="failed to get container status \"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42\": rpc error: code = NotFound desc = could not find container \"418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42\": container with ID starting with 418b92b5a5fca71afee093b36221623e09facd2086e88e8be8308ce4b6097b42 not found: ID does not exist" Jul 02 23:50:14.542704 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.542679 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:14.546410 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:14.546390 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8b3bd569-openshift-default-67c97b84f9-rtzj5"] Jul 02 23:50:16.432714 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:50:16.432672 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" path="/var/lib/kubelet/pods/4a1e69c0-fb60-4141-bbe6-d8370e1e505c/volumes" Jul 02 23:51:48.415636 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.415601 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:51:48.416209 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.416112 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerName="istio-proxy" Jul 02 23:51:48.416209 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.416130 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerName="istio-proxy" Jul 02 23:51:48.416329 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.416216 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a1e69c0-fb60-4141-bbe6-d8370e1e505c" containerName="istio-proxy" Jul 02 23:51:48.418889 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.418870 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.421307 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.421285 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f6ee9ad4-openshift-default-dockercfg-ppp56\"" Jul 02 23:51:48.438305 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.438282 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:51:48.576807 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.576987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.576987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.576987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.576987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.576987 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.576986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpdqv\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.577205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.577026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.577205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.577044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.577205 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.577066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.677933 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.677854 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.677933 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.677886 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.677945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpdqv\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.677972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.677995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678763 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.678868 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678844 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.679080 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.678966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.680615 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.680592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.681119 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.681102 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.689458 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.689438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.689823 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.689797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpdqv\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv\") pod \"e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.730786 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.730750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:48.868490 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.868469 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:51:48.872359 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:51:48.872320 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04000355_ffda_43b5_b40c_563d99399517.slice/crio-b13124d694d4a55a5551af1dbfd41e4a11aa671775a39f7a433c658b1226c93b WatchSource:0}: Error finding container b13124d694d4a55a5551af1dbfd41e4a11aa671775a39f7a433c658b1226c93b: Status 404 returned error can't find the container with id b13124d694d4a55a5551af1dbfd41e4a11aa671775a39f7a433c658b1226c93b Jul 02 23:51:48.875086 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.875049 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:51:48.875206 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.875134 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:51:48.875206 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.875178 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:51:48.928938 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:48.928870 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerStarted","Data":"b13124d694d4a55a5551af1dbfd41e4a11aa671775a39f7a433c658b1226c93b"} Jul 02 23:51:49.935251 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:49.935211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerStarted","Data":"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f"} Jul 02 23:51:49.954687 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:49.954634 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" podStartSLOduration=1.954615869 podStartE2EDuration="1.954615869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:51:49.952934177 +0000 UTC m=+1764.037535595" watchObservedRunningTime="2026-07-02 23:51:49.954615869 +0000 UTC m=+1764.039217287" Jul 02 23:51:50.731084 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:50.731041 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:50.736206 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:50.736180 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:50.939950 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:50.939918 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:51:50.940997 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:51:50.940980 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:06.025102 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:06.024725 2562 patch_prober.go:28] interesting pod/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:57978->10.132.0.34:15021: read: connection reset by peer" start-of-body= Jul 02 23:52:06.025102 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:06.024801 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:57978->10.132.0.34:15021: read: connection reset by peer" Jul 02 23:52:07.012582 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.012547 2562 generic.go:356] "Generic (PLEG): container finished" podID="04000355-ffda-43b5-b40c-563d99399517" containerID="7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f" exitCode=0 Jul 02 23:52:07.012794 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.012627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerDied","Data":"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f"} Jul 02 23:52:07.013151 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.013134 2562 scope.go:117] "RemoveContainer" containerID="7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f" Jul 02 23:52:07.014185 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.014153 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:07.014274 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.014242 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:07.014327 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:07.014273 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:08.019107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:08.019075 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerStarted","Data":"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b"} Jul 02 23:52:08.732024 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:08.731982 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:08.735441 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:08.735414 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:09.028824 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:09.025987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:09.029668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:09.029649 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:10.053663 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:10.053627 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:52:11.034688 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:11.034647 2562 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" secret="" err="secret \"e2e-mt-f6ee9ad4-openshift-default-dockercfg-ppp56\" not found" Jul 02 23:52:12.038951 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:12.038913 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" containerID="cri-o://5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b" gracePeriod=30 Jul 02 23:52:17.289834 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.289808 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:17.339404 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339318 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339404 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339366 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339404 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339401 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339430 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339457 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339496 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339551 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339594 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpdqv\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339633 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data\") pod \"04000355-ffda-43b5-b40c-563d99399517\" (UID: \"04000355-ffda-43b5-b40c-563d99399517\") " Jul 02 23:52:17.339935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339694 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:52:17.339935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339890 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:52:17.339935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339817 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:52:17.339935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.339913 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.340187 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.340160 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:52:17.340620 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.340602 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data" (OuterVolumeSpecName: "istio-data") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:52:17.341999 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.341958 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:52:17.342155 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.342132 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv" (OuterVolumeSpecName: "kube-api-access-mpdqv") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "kube-api-access-mpdqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:52:17.342249 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.342229 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:52:17.342516 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.342495 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token" (OuterVolumeSpecName: "istio-token") pod "04000355-ffda-43b5-b40c-563d99399517" (UID: "04000355-ffda-43b5-b40c-563d99399517"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:52:17.441365 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441328 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441365 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441356 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04000355-ffda-43b5-b40c-563d99399517-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441365 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441365 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpdqv\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-kube-api-access-mpdqv\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441365 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441376 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441384 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441392 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04000355-ffda-43b5-b40c-563d99399517-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441411 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04000355-ffda-43b5-b40c-563d99399517-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:17.441696 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:17.441419 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04000355-ffda-43b5-b40c-563d99399517-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:52:18.068028 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.067968 2562 generic.go:356] "Generic (PLEG): container finished" podID="04000355-ffda-43b5-b40c-563d99399517" containerID="5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b" exitCode=0 Jul 02 23:52:18.068191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.068038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerDied","Data":"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b"} Jul 02 23:52:18.068191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.068083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" event={"ID":"04000355-ffda-43b5-b40c-563d99399517","Type":"ContainerDied","Data":"b13124d694d4a55a5551af1dbfd41e4a11aa671775a39f7a433c658b1226c93b"} Jul 02 23:52:18.068191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.068103 2562 scope.go:117] "RemoveContainer" containerID="5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b" Jul 02 23:52:18.068191 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.068042 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7" Jul 02 23:52:18.076803 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.076786 2562 scope.go:117] "RemoveContainer" containerID="7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f" Jul 02 23:52:18.084830 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.084810 2562 scope.go:117] "RemoveContainer" containerID="5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b" Jul 02 23:52:18.085090 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:52:18.085072 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b\": container with ID starting with 5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b not found: ID does not exist" containerID="5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b" Jul 02 23:52:18.085345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.085100 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b"} err="failed to get container status \"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b\": rpc error: code = NotFound desc = could not find container \"5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b\": container with ID starting with 5d5e08889d4ffe8a8fa65d0d449ed7084fb1aeab23c9fe14adc377a5cf3bf97b not found: ID does not exist" Jul 02 23:52:18.085345 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.085122 2562 scope.go:117] "RemoveContainer" containerID="7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f" Jul 02 23:52:18.085345 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:52:18.085325 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f\": container with ID starting with 7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f not found: ID does not exist" containerID="7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f" Jul 02 23:52:18.085546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.085350 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f"} err="failed to get container status \"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f\": rpc error: code = NotFound desc = could not find container \"7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f\": container with ID starting with 7c07e911421baa4a5c4dddef5760cc746f7a4bec4d630f26de4e6cae22f5193f not found: ID does not exist" Jul 02 23:52:18.088137 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.088115 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:52:18.095186 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.095163 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f6ee9ad4-openshift-default-7b6d57c9f-ts8b7"] Jul 02 23:52:18.433659 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:18.433573 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04000355-ffda-43b5-b40c-563d99399517" path="/var/lib/kubelet/pods/04000355-ffda-43b5-b40c-563d99399517/volumes" Jul 02 23:52:29.216999 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.216922 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217266 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217278 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217291 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217296 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217354 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.217380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.217361 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="04000355-ffda-43b5-b40c-563d99399517" containerName="istio-proxy" Jul 02 23:52:29.219532 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.219512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.221900 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.221881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cadc708e-openshift-default-dockercfg-cw2wq\"" Jul 02 23:52:29.231645 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.231624 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:52:29.348648 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348648 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.348869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr28l\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.349095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.349095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.348921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450376 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr28l\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.450743 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.451065 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.451065 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.451065 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.450923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.451177 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.451162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.451240 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.451183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.453035 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.452998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.453251 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.453234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.458269 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.458241 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.458407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.458339 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr28l\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l\") pod \"e2e-mt-cadc708e-openshift-default-75b544ddcf-49459\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.529872 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.529837 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:29.665078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.665053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:52:29.667226 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:52:29.667200 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5537114_7c46_4b60_99e8_54c3043211cd.slice/crio-53d088990ceaf80837cdad8227842919129464f6e8da68cc475598b78f3f72cd WatchSource:0}: Error finding container 53d088990ceaf80837cdad8227842919129464f6e8da68cc475598b78f3f72cd: Status 404 returned error can't find the container with id 53d088990ceaf80837cdad8227842919129464f6e8da68cc475598b78f3f72cd Jul 02 23:52:29.669484 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.669453 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:29.669586 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.669531 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:29.669586 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:29.669577 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:30.122042 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:30.121981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerStarted","Data":"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2"} Jul 02 23:52:30.122217 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:30.122063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerStarted","Data":"53d088990ceaf80837cdad8227842919129464f6e8da68cc475598b78f3f72cd"} Jul 02 23:52:30.141454 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:30.141407 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podStartSLOduration=1.141392817 podStartE2EDuration="1.141392817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:52:30.14054844 +0000 UTC m=+1804.225149881" watchObservedRunningTime="2026-07-02 23:52:30.141392817 +0000 UTC m=+1804.225994290" Jul 02 23:52:30.530585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:30.530550 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:30.535981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:30.535954 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:31.125924 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:31.125888 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:31.126815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:31.126793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:45.186373 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.186338 2562 generic.go:356] "Generic (PLEG): container finished" podID="b5537114-7c46-4b60-99e8-54c3043211cd" containerID="a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2" exitCode=0 Jul 02 23:52:45.186758 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.186410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerDied","Data":"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2"} Jul 02 23:52:45.186811 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.186798 2562 scope.go:117] "RemoveContainer" containerID="a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2" Jul 02 23:52:45.187790 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.187761 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:45.187848 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.187831 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:45.187884 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:45.187871 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:52:46.126486 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:46.126459 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:46.192527 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:46.192496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerStarted","Data":"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be"} Jul 02 23:52:46.530094 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:46.530050 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:46.531581 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:46.531551 2562 patch_prober.go:28] interesting pod/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 02 23:52:46.531808 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:46.531781 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 02 23:52:47.530860 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:47.530818 2562 patch_prober.go:28] interesting pod/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 02 23:52:47.531477 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:47.531443 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 02 23:52:48.531313 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:48.531263 2562 patch_prober.go:28] interesting pod/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 02 23:52:48.531815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:48.531351 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 02 23:52:49.530736 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:49.530688 2562 patch_prober.go:28] interesting pod/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 02 23:52:49.530957 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:49.530773 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 02 23:52:50.534875 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:50.534842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:50.535327 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:50.535115 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:52:50.535710 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:52:50.535694 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:55:47.734573 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:47.734540 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:55:47.735104 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:47.734750 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" containerID="cri-o://3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be" gracePeriod=30 Jul 02 23:55:50.537876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:50.537833 2562 patch_prober.go:28] interesting pod/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 23:55:50.538275 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:50.537916 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 23:55:52.989819 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:52.989795 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:55:53.024533 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.024442 2562 generic.go:356] "Generic (PLEG): container finished" podID="b5537114-7c46-4b60-99e8-54c3043211cd" containerID="3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be" exitCode=0 Jul 02 23:55:53.024533 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.024500 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" Jul 02 23:55:53.024720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.024538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerDied","Data":"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be"} Jul 02 23:55:53.024720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.024607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459" event={"ID":"b5537114-7c46-4b60-99e8-54c3043211cd","Type":"ContainerDied","Data":"53d088990ceaf80837cdad8227842919129464f6e8da68cc475598b78f3f72cd"} Jul 02 23:55:53.024720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.024642 2562 scope.go:117] "RemoveContainer" containerID="3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be" Jul 02 23:55:53.040734 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.040709 2562 scope.go:117] "RemoveContainer" containerID="a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2" Jul 02 23:55:53.049400 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.049371 2562 scope.go:117] "RemoveContainer" containerID="3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be" Jul 02 23:55:53.049680 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:55:53.049658 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be\": container with ID starting with 3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be not found: ID does not exist" containerID="3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be" Jul 02 23:55:53.049741 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.049691 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be"} err="failed to get container status \"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be\": rpc error: code = NotFound desc = could not find container \"3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be\": container with ID starting with 3974a8ae2129120b541108cdc998551662e76c961ef871ab165eea220a0d59be not found: ID does not exist" Jul 02 23:55:53.049741 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.049711 2562 scope.go:117] "RemoveContainer" containerID="a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2" Jul 02 23:55:53.050036 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:55:53.049991 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2\": container with ID starting with a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2 not found: ID does not exist" containerID="a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2" Jul 02 23:55:53.050134 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.050043 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2"} err="failed to get container status \"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2\": rpc error: code = NotFound desc = could not find container \"a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2\": container with ID starting with a3b39c113f7abb2420b8c5845336d2d7570cff05f9d70d243270e05ff4990ae2 not found: ID does not exist" Jul 02 23:55:53.055286 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055266 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055320 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055356 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055372 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055405 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055388 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055469 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055505 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055536 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr28l\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l\") pod \"b5537114-7c46-4b60-99e8-54c3043211cd\" (UID: \"b5537114-7c46-4b60-99e8-54c3043211cd\") " Jul 02 23:55:53.055603 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055544 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:55:53.055835 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055709 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:55:53.055835 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055824 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.055940 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055831 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:55:53.055940 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.055848 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.056080 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.056037 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:55:53.056878 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.056845 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:55:53.058106 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.058076 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:55:53.058403 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.058364 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l" (OuterVolumeSpecName: "kube-api-access-dr28l") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "kube-api-access-dr28l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:55:53.058495 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.058470 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:55:53.058623 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.058603 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5537114-7c46-4b60-99e8-54c3043211cd" (UID: "b5537114-7c46-4b60-99e8-54c3043211cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:55:53.156864 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156830 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.156864 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156859 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.156864 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156868 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5537114-7c46-4b60-99e8-54c3043211cd-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.157126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156876 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.157126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156885 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5537114-7c46-4b60-99e8-54c3043211cd-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.157126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156892 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5537114-7c46-4b60-99e8-54c3043211cd-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.157126 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.156900 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr28l\" (UniqueName: \"kubernetes.io/projected/b5537114-7c46-4b60-99e8-54c3043211cd-kube-api-access-dr28l\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:55:53.346311 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.346274 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:55:53.352324 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:53.352295 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cadc708e-openshift-default-75b544ddcf-49459"] Jul 02 23:55:54.431890 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:55:54.431848 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" path="/var/lib/kubelet/pods/b5537114-7c46-4b60-99e8-54c3043211cd/volumes" Jul 02 23:56:41.891728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.891696 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:56:41.892147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.892029 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:41.892147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.892039 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:41.892147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.892125 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:41.892147 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.892133 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:41.894986 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.894966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.897520 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.897501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2b5047-openshift-default-dockercfg-wnl82\"" Jul 02 23:56:41.911876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.911844 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:56:41.971353 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95qp\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:41.971828 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:41.971583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072668 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95qp\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.072876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073240 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073240 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.072916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073412 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.073234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073412 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.073282 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073412 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.073361 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.073449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.073558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.073454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.075156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.075138 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.075593 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.075577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.081023 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.080989 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.081121 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.081092 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95qp\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp\") pod \"e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.205863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.205767 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:42.358738 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.358704 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:56:42.360591 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:56:42.360552 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0279e56a_b813_49b1_8087_0b389fdf9bbb.slice/crio-4aaf6155c47c091f61e72a253c5b55e3f35d9c51ac57abd5a4bc72c64593a266 WatchSource:0}: Error finding container 4aaf6155c47c091f61e72a253c5b55e3f35d9c51ac57abd5a4bc72c64593a266: Status 404 returned error can't find the container with id 4aaf6155c47c091f61e72a253c5b55e3f35d9c51ac57abd5a4bc72c64593a266 Jul 02 23:56:42.363129 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.363091 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:42.363214 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.363187 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:42.363257 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:42.363223 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:43.249851 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:43.249815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerStarted","Data":"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99"} Jul 02 23:56:43.250272 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:43.249875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerStarted","Data":"4aaf6155c47c091f61e72a253c5b55e3f35d9c51ac57abd5a4bc72c64593a266"} Jul 02 23:56:43.269945 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:43.269874 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podStartSLOduration=2.2698579260000002 podStartE2EDuration="2.269857926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:56:43.268603071 +0000 UTC m=+2057.353204550" watchObservedRunningTime="2026-07-02 23:56:43.269857926 +0000 UTC m=+2057.354459343" Jul 02 23:56:44.206713 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:44.206677 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:44.211697 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:44.211671 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:44.254676 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:44.254645 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:44.255511 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:44.255493 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:56:50.406755 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.406676 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:56:50.407107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.407043 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:50.407107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.407057 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5537114-7c46-4b60-99e8-54c3043211cd" containerName="istio-proxy" Jul 02 23:56:50.410475 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.410453 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.412765 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.412748 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6edf51-openshift-default-dockercfg-tg2m9\"" Jul 02 23:56:50.421896 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.421870 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:56:50.550841 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.550796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.550841 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.550843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.550860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.550951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.550984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.551104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.551127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.551174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.551386 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.551227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74cws\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652227 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652227 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652463 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652463 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652463 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652463 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652463 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652665 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652720 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652754 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74cws\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.652815 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.652791 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.653570 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.653545 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.654865 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.654837 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.655053 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.655037 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.661650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.661581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74cws\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.661650 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.661587 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token\") pod \"e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.723175 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.723146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:50.863065 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.863038 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:56:50.865530 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:56:50.865488 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77bebb10_abf1_42bb_be9e_88fa624e9ac7.slice/crio-869f0a9ee127e14dd6948e4f021a012d941416b82b94eb894c9feab9fd63d09a WatchSource:0}: Error finding container 869f0a9ee127e14dd6948e4f021a012d941416b82b94eb894c9feab9fd63d09a: Status 404 returned error can't find the container with id 869f0a9ee127e14dd6948e4f021a012d941416b82b94eb894c9feab9fd63d09a Jul 02 23:56:50.867716 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.867683 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:50.867805 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.867749 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:50.867805 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:50.867780 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:56:51.291195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:51.291156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerStarted","Data":"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429"} Jul 02 23:56:51.291389 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:51.291243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerStarted","Data":"869f0a9ee127e14dd6948e4f021a012d941416b82b94eb894c9feab9fd63d09a"} Jul 02 23:56:51.309935 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:51.309877 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podStartSLOduration=1.309861112 podStartE2EDuration="1.309861112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:56:51.309041415 +0000 UTC m=+2065.393642833" watchObservedRunningTime="2026-07-02 23:56:51.309861112 +0000 UTC m=+2065.394462530" Jul 02 23:56:51.724011 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:51.723977 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:51.729390 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:51.729366 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:52.297812 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:52.297778 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:56:52.298934 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:56:52.298914 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:57:00.288307 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.287960 2562 patch_prober.go:28] interesting pod/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 23:57:00.288307 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.288040 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 23:57:00.339990 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.339943 2562 generic.go:356] "Generic (PLEG): container finished" podID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerID="bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99" exitCode=0 Jul 02 23:57:00.340195 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.340143 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerDied","Data":"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99"} Jul 02 23:57:00.341103 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.340910 2562 scope.go:117] "RemoveContainer" containerID="bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99" Jul 02 23:57:00.343670 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.342200 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:00.343670 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.342287 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:00.343670 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:00.342338 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:01.350416 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:01.350366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerStarted","Data":"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360"} Jul 02 23:57:02.206715 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:02.206680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:57:02.206896 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:02.206871 2562 patch_prober.go:28] interesting pod/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 23:57:02.206942 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:02.206917 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 23:57:03.206593 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:03.206551 2562 patch_prober.go:28] interesting pod/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 23:57:03.207035 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:03.206631 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 23:57:04.206275 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:04.206228 2562 patch_prober.go:28] interesting pod/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 23:57:04.206428 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:04.206310 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 23:57:05.206537 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:05.206490 2562 patch_prober.go:28] interesting pod/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 02 23:57:05.206921 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:05.206580 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 02 23:57:06.210274 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:06.210240 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:57:06.210731 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:06.210639 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:57:06.211403 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:06.211381 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:57:07.299522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.299489 2562 patch_prober.go:28] interesting pod/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 02 23:57:07.299938 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.299553 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 02 23:57:07.383981 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.383952 2562 generic.go:356] "Generic (PLEG): container finished" podID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerID="0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429" exitCode=0 Jul 02 23:57:07.384176 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.384035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerDied","Data":"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429"} Jul 02 23:57:07.384547 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.384528 2562 scope.go:117] "RemoveContainer" containerID="0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429" Jul 02 23:57:07.385490 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.385444 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:07.385568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.385506 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:07.385568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:07.385545 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:57:08.390820 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:08.390786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerStarted","Data":"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc"} Jul 02 23:57:08.723866 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:08.723757 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:57:08.724210 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:08.724186 2562 patch_prober.go:28] interesting pod/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 02 23:57:08.724380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:08.724355 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 02 23:57:09.724062 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:09.724023 2562 patch_prober.go:28] interesting pod/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 02 23:57:09.724661 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:09.724628 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 02 23:57:10.723979 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:10.723930 2562 patch_prober.go:28] interesting pod/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 02 23:57:10.724216 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:10.724026 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 02 23:57:11.724600 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:11.724549 2562 patch_prober.go:28] interesting pod/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 02 23:57:11.725130 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:11.724624 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 02 23:57:12.727742 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:12.727709 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:57:12.728154 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:12.728035 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:57:12.728873 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:57:12.728853 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:58:40.435097 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:40.435062 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:58:40.435566 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:40.435310 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" containerID="cri-o://2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360" gracePeriod=30 Jul 02 23:58:45.687394 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.687371 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:58:45.709106 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709035 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709106 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709080 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709110 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j95qp\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709318 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709280 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709363 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709317 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709420 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709381 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709420 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709402 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709521 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709398 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:45.709521 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709447 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709521 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709486 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs\") pod \"0279e56a-b813-49b1-8087-0b389fdf9bbb\" (UID: \"0279e56a-b813-49b1-8087-0b389fdf9bbb\") " Jul 02 23:58:45.709675 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709547 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:45.709820 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709794 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.709980 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.709965 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.710264 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.710235 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:45.710522 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.710312 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:58:45.711263 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.711225 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:45.711951 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.711917 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:58:45.711951 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.711931 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp" (OuterVolumeSpecName: "kube-api-access-j95qp") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "kube-api-access-j95qp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:45.711951 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.711933 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:45.712283 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.712265 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "0279e56a-b813-49b1-8087-0b389fdf9bbb" (UID: "0279e56a-b813-49b1-8087-0b389fdf9bbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:45.811060 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811032 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811060 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811059 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811071 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811082 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j95qp\" (UniqueName: \"kubernetes.io/projected/0279e56a-b813-49b1-8087-0b389fdf9bbb-kube-api-access-j95qp\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811092 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0279e56a-b813-49b1-8087-0b389fdf9bbb-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811103 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.811277 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.811112 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0279e56a-b813-49b1-8087-0b389fdf9bbb-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:45.812271 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.812246 2562 generic.go:356] "Generic (PLEG): container finished" podID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerID="2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360" exitCode=0 Jul 02 23:58:45.812396 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.812309 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" Jul 02 23:58:45.812396 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.812332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerDied","Data":"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360"} Jul 02 23:58:45.812396 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.812384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd" event={"ID":"0279e56a-b813-49b1-8087-0b389fdf9bbb","Type":"ContainerDied","Data":"4aaf6155c47c091f61e72a253c5b55e3f35d9c51ac57abd5a4bc72c64593a266"} Jul 02 23:58:45.812521 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.812406 2562 scope.go:117] "RemoveContainer" containerID="2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360" Jul 02 23:58:45.821294 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.821280 2562 scope.go:117] "RemoveContainer" containerID="bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99" Jul 02 23:58:45.828607 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.828591 2562 scope.go:117] "RemoveContainer" containerID="2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360" Jul 02 23:58:45.828841 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:45.828824 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360\": container with ID starting with 2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360 not found: ID does not exist" containerID="2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360" Jul 02 23:58:45.828899 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.828846 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360"} err="failed to get container status \"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360\": rpc error: code = NotFound desc = could not find container \"2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360\": container with ID starting with 2a4ab1d6fe881827613fc43b5a056c7d8713b3ecb493c21bd6fb03e857d9d360 not found: ID does not exist" Jul 02 23:58:45.828899 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.828860 2562 scope.go:117] "RemoveContainer" containerID="bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99" Jul 02 23:58:45.829073 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:45.829054 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99\": container with ID starting with bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99 not found: ID does not exist" containerID="bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99" Jul 02 23:58:45.829116 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.829079 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99"} err="failed to get container status \"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99\": rpc error: code = NotFound desc = could not find container \"bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99\": container with ID starting with bb82bf59d0d870a8ee2dda080fcc91434ae8215b4336ffa564a6426826026c99 not found: ID does not exist" Jul 02 23:58:45.832906 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.832885 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:58:45.836175 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:45.836156 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2b5047-openshift-default-5959c994b5-mm8gd"] Jul 02 23:58:46.437556 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:46.437524 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" path="/var/lib/kubelet/pods/0279e56a-b813-49b1-8087-0b389fdf9bbb/volumes" Jul 02 23:58:47.224983 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:47.224946 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:58:47.225375 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:47.225255 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" containerID="cri-o://abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc" gracePeriod=30 Jul 02 23:58:52.478819 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.478795 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:58:52.572931 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.572845 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.572931 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.572886 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.572931 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.572915 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.572939 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74cws\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.572999 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573079 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573102 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573136 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573254 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573166 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo\") pod \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\" (UID: \"77bebb10-abf1-42bb-be9e-88fa624e9ac7\") " Jul 02 23:58:52.573556 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573383 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:52.573556 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573479 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.573556 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573387 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:58:52.573716 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573640 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:52.573716 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573680 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:52.573876 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.573841 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data" (OuterVolumeSpecName: "istio-data") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:52.575300 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.575277 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:58:52.575629 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.575610 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:58:52.575869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.575855 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token" (OuterVolumeSpecName: "istio-token") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:52.575869 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.575854 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws" (OuterVolumeSpecName: "kube-api-access-74cws") pod "77bebb10-abf1-42bb-be9e-88fa624e9ac7" (UID: "77bebb10-abf1-42bb-be9e-88fa624e9ac7"). InnerVolumeSpecName "kube-api-access-74cws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:58:52.674695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674651 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.674695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674686 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.674695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674695 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.674695 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674704 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.675043 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674735 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74cws\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-kube-api-access-74cws\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.675043 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674745 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.675043 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674753 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77bebb10-abf1-42bb-be9e-88fa624e9ac7-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.675043 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.674760 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77bebb10-abf1-42bb-be9e-88fa624e9ac7-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:58:52.845166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.845075 2562 generic.go:356] "Generic (PLEG): container finished" podID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerID="abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc" exitCode=0 Jul 02 23:58:52.845166 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.845138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerDied","Data":"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc"} Jul 02 23:58:52.845342 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.845209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" event={"ID":"77bebb10-abf1-42bb-be9e-88fa624e9ac7","Type":"ContainerDied","Data":"869f0a9ee127e14dd6948e4f021a012d941416b82b94eb894c9feab9fd63d09a"} Jul 02 23:58:52.845342 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.845139 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6" Jul 02 23:58:52.845342 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.845241 2562 scope.go:117] "RemoveContainer" containerID="abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc" Jul 02 23:58:52.855822 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.855800 2562 scope.go:117] "RemoveContainer" containerID="0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429" Jul 02 23:58:52.864075 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.864047 2562 scope.go:117] "RemoveContainer" containerID="abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc" Jul 02 23:58:52.864321 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:52.864302 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc\": container with ID starting with abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc not found: ID does not exist" containerID="abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc" Jul 02 23:58:52.864380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.864331 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc"} err="failed to get container status \"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc\": rpc error: code = NotFound desc = could not find container \"abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc\": container with ID starting with abeb60ebc9a869ebbef552bf76d0b6369b777ad3712a34904a738796d36eadbc not found: ID does not exist" Jul 02 23:58:52.864380 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.864348 2562 scope.go:117] "RemoveContainer" containerID="0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429" Jul 02 23:58:52.864565 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:52.864546 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429\": container with ID starting with 0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429 not found: ID does not exist" containerID="0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429" Jul 02 23:58:52.864619 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.864574 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429"} err="failed to get container status \"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429\": rpc error: code = NotFound desc = could not find container \"0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429\": container with ID starting with 0b5b87a3e9a83865ba7e3997edd9b187fe4dd01f8a3ab1381dec73caba583429 not found: ID does not exist" Jul 02 23:58:52.867795 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.867773 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:58:52.871716 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:52.871695 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6edf51-openshift-default-657755c865-dnpv6"] Jul 02 23:58:54.432956 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.432926 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" path="/var/lib/kubelet/pods/77bebb10-abf1-42bb-be9e-88fa624e9ac7/volumes" Jul 02 23:58:54.473049 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.472991 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:58:54.473383 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473370 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.473433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473386 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.473433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473393 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:54.473433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473399 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:54.473433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473413 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.473433 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473422 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.473597 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473480 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.473597 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473489 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:54.473597 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.473498 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0279e56a-b813-49b1-8087-0b389fdf9bbb" containerName="istio-proxy" Jul 02 23:58:54.476801 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.476781 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.479287 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.479263 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-a89e2b-serving-cert\"" Jul 02 23:58:54.479419 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.479401 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mfwkn\"" Jul 02 23:58:54.486787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.486761 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:58:54.591960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.591857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.591960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.591917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzmvz\" (UniqueName: \"kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.592270 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.591974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.693558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.693462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzmvz\" (UniqueName: \"kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.693558 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.693515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.693801 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.693580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.693801 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:54.693688 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-a89e2b-serving-cert: secret "maas-api-e2e-api-a-a89e2b-serving-cert" not found Jul 02 23:58:54.693801 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:58:54.693767 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls podName:8e6228c7-2c21-4840-b081-7026b45cd930 nodeName:}" failed. No retries permitted until 2026-07-02 23:58:55.193750351 +0000 UTC m=+2189.278351750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls") pod "maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" (UID: "8e6228c7-2c21-4840-b081-7026b45cd930") : secret "maas-api-e2e-api-a-a89e2b-serving-cert" not found Jul 02 23:58:54.694196 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.694180 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:54.703995 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:54.703963 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzmvz\" (UniqueName: \"kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:55.197236 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.197199 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:55.199812 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.199784 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") pod \"maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:55.387568 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.387523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:55.523707 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.523683 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:58:55.525449 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:58:55.525421 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6228c7_2c21_4840_b081_7026b45cd930.slice/crio-6d87cc42e22d1c9f8bd548d444e1ab0f10a16e976b4ba933ac08e99b3701e61e WatchSource:0}: Error finding container 6d87cc42e22d1c9f8bd548d444e1ab0f10a16e976b4ba933ac08e99b3701e61e: Status 404 returned error can't find the container with id 6d87cc42e22d1c9f8bd548d444e1ab0f10a16e976b4ba933ac08e99b3701e61e Jul 02 23:58:55.526828 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.526808 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:58:55.859809 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:55.859709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" event={"ID":"8e6228c7-2c21-4840-b081-7026b45cd930","Type":"ContainerStarted","Data":"6d87cc42e22d1c9f8bd548d444e1ab0f10a16e976b4ba933ac08e99b3701e61e"} Jul 02 23:58:58.877714 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:58.877676 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" event={"ID":"8e6228c7-2c21-4840-b081-7026b45cd930","Type":"ContainerStarted","Data":"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a"} Jul 02 23:58:58.878153 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:58.877841 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:58:58.894110 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:58.894064 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" podStartSLOduration=2.375026047 podStartE2EDuration="4.894049126s" totalImagesPullingTime="2.519023079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:58:58.892764982 +0000 UTC m=+2192.977366391" watchObservedRunningTime="2026-07-02 23:58:58.894049126 +0000 UTC m=+2192.978650605" Jul 02 23:58:59.659536 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.659500 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:58:59.659823 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.659810 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:59.659823 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.659824 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:59.659920 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.659893 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77bebb10-abf1-42bb-be9e-88fa624e9ac7" containerName="istio-proxy" Jul 02 23:58:59.662955 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.662935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.665419 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.665400 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-18475e-openshift-default-dockercfg-vpl68\"" Jul 02 23:58:59.674863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.674800 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:58:59.740152 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7mh\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740159 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740190 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740246 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740343 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740543 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.740543 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.740406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.841819 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.841819 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.841938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7mh\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842114 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842345 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842378 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842561 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842561 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842505 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.842893 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.842868 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.844351 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.844331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.844631 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.844613 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.850281 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.850252 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.850375 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.850293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7mh\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh\") pod \"e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:58:59.973394 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:58:59.973362 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:00.112845 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.112658 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:59:00.115565 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:59:00.115536 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6f4ca4_7f65_424f_b473_25dc20a68889.slice/crio-da233c58296a2dccc2cdb22356ea43f8d1c88f0899b51dda9188c01e79a3cafe WatchSource:0}: Error finding container da233c58296a2dccc2cdb22356ea43f8d1c88f0899b51dda9188c01e79a3cafe: Status 404 returned error can't find the container with id da233c58296a2dccc2cdb22356ea43f8d1c88f0899b51dda9188c01e79a3cafe Jul 02 23:59:00.118177 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.118137 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:00.118279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.118221 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:00.118279 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.118262 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:00.887960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.887925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerStarted","Data":"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218"} Jul 02 23:59:00.888170 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.887980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerStarted","Data":"da233c58296a2dccc2cdb22356ea43f8d1c88f0899b51dda9188c01e79a3cafe"} Jul 02 23:59:00.907104 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.907051 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podStartSLOduration=1.907035995 podStartE2EDuration="1.907035995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:59:00.905194256 +0000 UTC m=+2194.989795676" watchObservedRunningTime="2026-07-02 23:59:00.907035995 +0000 UTC m=+2194.991637410" Jul 02 23:59:00.974466 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:00.974433 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:01.979393 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:01.979368 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:02.897170 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:02.897131 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:02.898121 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:02.898103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:04.887176 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:04.887149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:59:15.961927 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.961892 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerID="c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218" exitCode=0 Jul 02 23:59:15.962374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.961969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerDied","Data":"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218"} Jul 02 23:59:15.962374 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.962358 2562 scope.go:117] "RemoveContainer" containerID="c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218" Jul 02 23:59:15.963460 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.963424 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:15.963569 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.963517 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:15.963569 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.963562 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 23:59:15.987472 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.987429 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:59:15.987960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:15.987930 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" podUID="8e6228c7-2c21-4840-b081-7026b45cd930" containerName="maas-api" containerID="cri-o://a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a" gracePeriod=30 Jul 02 23:59:16.242164 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.242133 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:59:16.402647 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.402596 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzmvz\" (UniqueName: \"kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz\") pod \"8e6228c7-2c21-4840-b081-7026b45cd930\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " Jul 02 23:59:16.402832 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.402724 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") pod \"8e6228c7-2c21-4840-b081-7026b45cd930\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " Jul 02 23:59:16.402832 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.402788 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca\") pod \"8e6228c7-2c21-4840-b081-7026b45cd930\" (UID: \"8e6228c7-2c21-4840-b081-7026b45cd930\") " Jul 02 23:59:16.403213 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.403178 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e6228c7-2c21-4840-b081-7026b45cd930" (UID: "8e6228c7-2c21-4840-b081-7026b45cd930"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:59:16.404945 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.404918 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e6228c7-2c21-4840-b081-7026b45cd930" (UID: "8e6228c7-2c21-4840-b081-7026b45cd930"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:59:16.405078 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.404945 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz" (OuterVolumeSpecName: "kube-api-access-mzmvz") pod "8e6228c7-2c21-4840-b081-7026b45cd930" (UID: "8e6228c7-2c21-4840-b081-7026b45cd930"). InnerVolumeSpecName "kube-api-access-mzmvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:59:16.503552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.503467 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6228c7-2c21-4840-b081-7026b45cd930-openshift-service-ca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:16.503552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.503497 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzmvz\" (UniqueName: \"kubernetes.io/projected/8e6228c7-2c21-4840-b081-7026b45cd930-kube-api-access-mzmvz\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:16.503552 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.503508 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6228c7-2c21-4840-b081-7026b45cd930-maas-api-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:16.967750 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.967660 2562 generic.go:356] "Generic (PLEG): container finished" podID="8e6228c7-2c21-4840-b081-7026b45cd930" containerID="a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a" exitCode=0 Jul 02 23:59:16.967750 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.967706 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" Jul 02 23:59:16.968303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.967751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" event={"ID":"8e6228c7-2c21-4840-b081-7026b45cd930","Type":"ContainerDied","Data":"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a"} Jul 02 23:59:16.968303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.967802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4" event={"ID":"8e6228c7-2c21-4840-b081-7026b45cd930","Type":"ContainerDied","Data":"6d87cc42e22d1c9f8bd548d444e1ab0f10a16e976b4ba933ac08e99b3701e61e"} Jul 02 23:59:16.968303 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.967834 2562 scope.go:117] "RemoveContainer" containerID="a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a" Jul 02 23:59:16.969813 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.969787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerStarted","Data":"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f"} Jul 02 23:59:16.973672 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.973642 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:16.977377 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.977354 2562 scope.go:117] "RemoveContainer" containerID="a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a" Jul 02 23:59:16.977723 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:16.977697 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a\": container with ID starting with a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a not found: ID does not exist" containerID="a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a" Jul 02 23:59:16.977794 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.977737 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a"} err="failed to get container status \"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a\": rpc error: code = NotFound desc = could not find container \"a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a\": container with ID starting with a650a353bebe3a33e00d87bc395b02091f825d74442889fe4a3cf161d6f5f76a not found: ID does not exist" Jul 02 23:59:16.983505 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.983478 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:59:16.986819 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:16.986798 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a89e2b-7fc7c77df4-xjwv4"] Jul 02 23:59:17.898120 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:17.898083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:17.975585 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:17.975548 2562 patch_prober.go:28] interesting pod/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 02 23:59:17.975953 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:17.975618 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 02 23:59:18.435863 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:18.435817 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e6228c7-2c21-4840-b081-7026b45cd930" path="/var/lib/kubelet/pods/8e6228c7-2c21-4840-b081-7026b45cd930/volumes" Jul 02 23:59:18.974546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:18.974511 2562 patch_prober.go:28] interesting pod/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 02 23:59:18.974728 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:18.974568 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 02 23:59:19.974485 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:19.974450 2562 patch_prober.go:28] interesting pod/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 02 23:59:19.974848 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:19.974516 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 02 23:59:20.974798 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:20.974753 2562 patch_prober.go:28] interesting pod/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 02 23:59:20.975218 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:20.974819 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 02 23:59:21.984156 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:21.984120 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:21.985632 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:21.985601 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:28.584243 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:28.584207 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:59:28.584707 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:28.584453 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" containerID="cri-o://da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f" gracePeriod=30 Jul 02 23:59:32.715407 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.715376 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 02 23:59:32.715778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.715718 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e6228c7-2c21-4840-b081-7026b45cd930" containerName="maas-api" Jul 02 23:59:32.715778 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.715732 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e6228c7-2c21-4840-b081-7026b45cd930" containerName="maas-api" Jul 02 23:59:32.715851 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.715793 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e6228c7-2c21-4840-b081-7026b45cd930" containerName="maas-api" Jul 02 23:59:32.718607 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.718591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.720972 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.720949 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-c1893b-serving-cert\"" Jul 02 23:59:32.721150 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.721027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mfwkn\"" Jul 02 23:59:32.727265 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.727239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 02 23:59:32.757177 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.757038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.757503 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.757484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.757697 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.757667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwhb4\" (UniqueName: \"kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.858546 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.858511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.858756 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.858574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.858756 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.858621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwhb4\" (UniqueName: \"kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.858756 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:32.858672 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-c1893b-serving-cert: secret "maas-api-e2e-shared-a-c1893b-serving-cert" not found Jul 02 23:59:32.858756 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:32.858736 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls podName:533ca191-7818-4a22-a9fb-d7deae542b7e nodeName:}" failed. No retries permitted until 2026-07-02 23:59:33.358720321 +0000 UTC m=+2227.443321717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls") pod "maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" (UID: "533ca191-7818-4a22-a9fb-d7deae542b7e") : secret "maas-api-e2e-shared-a-c1893b-serving-cert" not found Jul 02 23:59:32.859247 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.859228 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:32.866637 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:32.866616 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwhb4\" (UniqueName: \"kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:33.363855 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.363819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:33.366555 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.366526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:33.629651 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.629577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:33.769850 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.769818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 02 23:59:33.772664 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:59:33.772631 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533ca191_7818_4a22_a9fb_d7deae542b7e.slice/crio-f33a71f9eb5c8e04f637f7e36cc81b00cee97d76dfda498079af675690a3f21f WatchSource:0}: Error finding container f33a71f9eb5c8e04f637f7e36cc81b00cee97d76dfda498079af675690a3f21f: Status 404 returned error can't find the container with id f33a71f9eb5c8e04f637f7e36cc81b00cee97d76dfda498079af675690a3f21f Jul 02 23:59:33.857780 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.857751 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:33.968611 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968524 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968611 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968572 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968611 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968609 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968674 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq7mh\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968690 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968741 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968778 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.968887 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.968800 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data\") pod \"4b6f4ca4-7f65-424f-b473-25dc20a68889\" (UID: \"4b6f4ca4-7f65-424f-b473-25dc20a68889\") " Jul 02 23:59:33.969250 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.969114 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:59:33.969250 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.969130 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:59:33.969250 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.969179 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:59:33.969571 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.969314 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:59:33.969792 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.969773 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:59:33.971000 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.970971 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:59:33.971000 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.971029 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 23:59:33.971492 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.971471 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh" (OuterVolumeSpecName: "kube-api-access-rq7mh") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "kube-api-access-rq7mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:59:33.971560 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:33.971476 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b6f4ca4-7f65-424f-b473-25dc20a68889" (UID: "4b6f4ca4-7f65-424f-b473-25dc20a68889"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:59:34.050605 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.050571 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerID="da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f" exitCode=0 Jul 02 23:59:34.050789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.050625 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" Jul 02 23:59:34.050789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.050656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerDied","Data":"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f"} Jul 02 23:59:34.050789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.050712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq" event={"ID":"4b6f4ca4-7f65-424f-b473-25dc20a68889","Type":"ContainerDied","Data":"da233c58296a2dccc2cdb22356ea43f8d1c88f0899b51dda9188c01e79a3cafe"} Jul 02 23:59:34.050789 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.050741 2562 scope.go:117] "RemoveContainer" containerID="da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f" Jul 02 23:59:34.051960 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.051931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" event={"ID":"533ca191-7818-4a22-a9fb-d7deae542b7e","Type":"ContainerStarted","Data":"f33a71f9eb5c8e04f637f7e36cc81b00cee97d76dfda498079af675690a3f21f"} Jul 02 23:59:34.059524 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.059506 2562 scope.go:117] "RemoveContainer" containerID="c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218" Jul 02 23:59:34.068111 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.068088 2562 scope.go:117] "RemoveContainer" containerID="da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f" Jul 02 23:59:34.068385 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:34.068360 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f\": container with ID starting with da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f not found: ID does not exist" containerID="da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f" Jul 02 23:59:34.068461 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.068398 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f"} err="failed to get container status \"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f\": rpc error: code = NotFound desc = could not find container \"da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f\": container with ID starting with da092d10500383d269f0e86144393ca17f11200383a7ada63113ff8679da649f not found: ID does not exist" Jul 02 23:59:34.068461 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.068423 2562 scope.go:117] "RemoveContainer" containerID="c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218" Jul 02 23:59:34.068694 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:34.068671 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218\": container with ID starting with c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218 not found: ID does not exist" containerID="c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218" Jul 02 23:59:34.068741 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.068701 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218"} err="failed to get container status \"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218\": rpc error: code = NotFound desc = could not find container \"c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218\": container with ID starting with c5ae02b31069759a640a7e9ee5bb5322d7bb2a3c7c10f27f5c3f7382ccb82218 not found: ID does not exist" Jul 02 23:59:34.069687 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069666 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6f4ca4-7f65-424f-b473-25dc20a68889-istiod-ca-cert\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069687 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069685 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-token\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069695 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq7mh\" (UniqueName: \"kubernetes.io/projected/4b6f4ca4-7f65-424f-b473-25dc20a68889-kube-api-access-rq7mh\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069706 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069716 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-credential-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069725 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-envoy\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069733 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-data\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069748 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6f4ca4-7f65-424f-b473-25dc20a68889-workload-socket\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.069847 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.069759 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6f4ca4-7f65-424f-b473-25dc20a68889-istio-podinfo\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 02 23:59:34.071394 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.071375 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:59:34.075030 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.074991 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-18475e-openshift-default-68956ffbb9-4lqsq"] Jul 02 23:59:34.432531 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:34.432493 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" path="/var/lib/kubelet/pods/4b6f4ca4-7f65-424f-b473-25dc20a68889/volumes" Jul 02 23:59:35.059167 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:35.059133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" event={"ID":"533ca191-7818-4a22-a9fb-d7deae542b7e","Type":"ContainerStarted","Data":"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9"} Jul 02 23:59:35.059656 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:35.059234 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:35.076040 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:35.075971 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" podStartSLOduration=2.6596476239999998 podStartE2EDuration="3.075952357s" totalImagesPullingTime="416.304733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:59:35.073453172 +0000 UTC m=+2229.158054603" watchObservedRunningTime="2026-07-02 23:59:35.075952357 +0000 UTC m=+2229.160553775" Jul 02 23:59:40.862906 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.862870 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 02 23:59:40.863455 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.863430 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 02 23:59:40.863616 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.863595 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 02 23:59:40.863723 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.863620 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 02 23:59:40.863723 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.863629 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 02 23:59:40.863811 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.863726 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 02 23:59:40.867083 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.867055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:40.869334 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.869311 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-c04a64-serving-cert\"" Jul 02 23:59:40.878787 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.878755 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 02 23:59:40.930596 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.930556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj824\" (UniqueName: \"kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:40.930806 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.930619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:40.930806 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:40.930650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.031954 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.031914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj824\" (UniqueName: \"kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.032162 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.031980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.032162 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.032001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.032162 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:41.032126 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-c04a64-serving-cert: secret "maas-api-e2e-shared-b-c04a64-serving-cert" not found Jul 02 23:59:41.032272 ip-10-0-130-160 kubenswrapper[2562]: E0702 23:59:41.032202 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls podName:e8553366-6e6b-4c9c-841d-a27560e73ce5 nodeName:}" failed. No retries permitted until 2026-07-02 23:59:41.532181848 +0000 UTC m=+2235.616783258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls") pod "maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" (UID: "e8553366-6e6b-4c9c-841d-a27560e73ce5") : secret "maas-api-e2e-shared-b-c04a64-serving-cert" not found Jul 02 23:59:41.032780 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.032760 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.040631 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.040595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj824\" (UniqueName: \"kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.069584 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.069554 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 02 23:59:41.536259 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.536219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.538781 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.538756 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") pod \"maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.780188 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.780129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:41.918113 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:41.918086 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 02 23:59:41.920726 ip-10-0-130-160 kubenswrapper[2562]: W0702 23:59:41.920699 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8553366_6e6b_4c9c_841d_a27560e73ce5.slice/crio-43fe92741709c3f757062b0bc8def51bc6186d11fbfb404878dc04790e9fe090 WatchSource:0}: Error finding container 43fe92741709c3f757062b0bc8def51bc6186d11fbfb404878dc04790e9fe090: Status 404 returned error can't find the container with id 43fe92741709c3f757062b0bc8def51bc6186d11fbfb404878dc04790e9fe090 Jul 02 23:59:42.092095 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:42.091989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" event={"ID":"e8553366-6e6b-4c9c-841d-a27560e73ce5","Type":"ContainerStarted","Data":"43fe92741709c3f757062b0bc8def51bc6186d11fbfb404878dc04790e9fe090"} Jul 02 23:59:43.098137 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:43.098104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" event={"ID":"e8553366-6e6b-4c9c-841d-a27560e73ce5","Type":"ContainerStarted","Data":"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b"} Jul 02 23:59:43.098559 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:43.098168 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 02 23:59:43.113107 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:43.113061 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" podStartSLOduration=2.791916392 podStartE2EDuration="3.113045179s" totalImagesPullingTime="321.128787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:59:43.111851518 +0000 UTC m=+2237.196452937" watchObservedRunningTime="2026-07-02 23:59:43.113045179 +0000 UTC m=+2237.197646596" Jul 02 23:59:49.108339 ip-10-0-130-160 kubenswrapper[2562]: I0702 23:59:49.108264 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 03 00:00:00.127402 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.127367 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888"] Jul 03 00:00:00.127845 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.127781 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6f4ca4-7f65-424f-b473-25dc20a68889" containerName="istio-proxy" Jul 03 00:00:00.130607 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.130585 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717280-x9f86"] Jul 03 00:00:00.130723 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.130709 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:00.133658 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.133641 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267"] Jul 03 00:00:00.133787 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.133769 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:00.136744 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.136723 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29717280-kqc4w"] Jul 03 00:00:00.136920 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.136813 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:00.140274 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.140248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888"] Jul 03 00:00:00.140396 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.140339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.142790 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.142771 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-qqffs\"" Jul 03 00:00:00.143104 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.142819 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Jul 03 00:00:00.144177 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.144124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267"] Jul 03 00:00:00.147160 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.146565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717280-x9f86"] Jul 03 00:00:00.148709 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.148685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29717280-kqc4w"] Jul 03 00:00:00.307021 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.306982 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x28zq\" (UniqueName: \"kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq\") pod \"maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888\" (UID: \"af0b0d68-206d-4d45-b0c3-aa11851d9aa7\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:00.307178 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.307038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsr84\" (UniqueName: \"kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84\") pod \"maas-api-key-cleanup-29717280-x9f86\" (UID: \"a195f5b6-96f5-4341-abec-2fecdfce52ae\") " pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:00.307178 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.307095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5nj\" (UniqueName: \"kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj\") pod \"maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267\" (UID: \"26c1d823-bd8a-491b-8726-b569322be15c\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:00.307178 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.307126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ks48\" (UniqueName: \"kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.307289 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.307258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.408736 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.408657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x28zq\" (UniqueName: \"kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq\") pod \"maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888\" (UID: \"af0b0d68-206d-4d45-b0c3-aa11851d9aa7\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:00.408736 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.408699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsr84\" (UniqueName: \"kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84\") pod \"maas-api-key-cleanup-29717280-x9f86\" (UID: \"a195f5b6-96f5-4341-abec-2fecdfce52ae\") " pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:00.408736 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.408721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5nj\" (UniqueName: \"kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj\") pod \"maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267\" (UID: \"26c1d823-bd8a-491b-8726-b569322be15c\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:00.409017 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.408747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ks48\" (UniqueName: \"kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.409017 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.408839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.409481 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.409461 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.417072 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.417043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ks48\" (UniqueName: \"kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48\") pod \"image-pruner-29717280-kqc4w\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.417259 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.417236 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsr84\" (UniqueName: \"kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84\") pod \"maas-api-key-cleanup-29717280-x9f86\" (UID: \"a195f5b6-96f5-4341-abec-2fecdfce52ae\") " pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:00.417632 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.417608 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5nj\" (UniqueName: \"kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj\") pod \"maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267\" (UID: \"26c1d823-bd8a-491b-8726-b569322be15c\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:00.417706 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.417608 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x28zq\" (UniqueName: \"kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq\") pod \"maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888\" (UID: \"af0b0d68-206d-4d45-b0c3-aa11851d9aa7\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:00.457847 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.457821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:00.466847 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.466819 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:00.474798 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.474736 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:00.481130 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.480780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:00.838132 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.838103 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888"] Jul 03 00:00:00.840284 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.840260 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717280-x9f86"] Jul 03 00:00:00.840638 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:00:00.840614 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda195f5b6_96f5_4341_abec_2fecdfce52ae.slice/crio-86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd WatchSource:0}: Error finding container 86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd: Status 404 returned error can't find the container with id 86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd Jul 03 00:00:00.842593 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:00:00.842575 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf0b0d68_206d_4d45_b0c3_aa11851d9aa7.slice/crio-4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243 WatchSource:0}: Error finding container 4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243: Status 404 returned error can't find the container with id 4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243 Jul 03 00:00:00.880632 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.880603 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267"] Jul 03 00:00:00.882834 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:00.882809 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29717280-kqc4w"] Jul 03 00:00:00.883056 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:00:00.883036 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7523c65e_d9d6_4bc0_b225_43c696b221e7.slice/crio-9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d WatchSource:0}: Error finding container 9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d: Status 404 returned error can't find the container with id 9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d Jul 03 00:00:00.883970 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:00:00.883947 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c1d823_bd8a_491b_8726_b569322be15c.slice/crio-dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a WatchSource:0}: Error finding container dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a: Status 404 returned error can't find the container with id dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a Jul 03 00:00:01.181981 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.181876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29717280-kqc4w" event={"ID":"7523c65e-d9d6-4bc0-b225-43c696b221e7","Type":"ContainerStarted","Data":"2a37cc02fef992007c4cb7232b93181b9c816a4c06ca17aa1babec966cbd8688"} Jul 03 00:00:01.181981 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.181934 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29717280-kqc4w" event={"ID":"7523c65e-d9d6-4bc0-b225-43c696b221e7","Type":"ContainerStarted","Data":"9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d"} Jul 03 00:00:01.184107 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.184083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" event={"ID":"26c1d823-bd8a-491b-8726-b569322be15c","Type":"ContainerStarted","Data":"dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a"} Jul 03 00:00:01.185405 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.185381 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" event={"ID":"a195f5b6-96f5-4341-abec-2fecdfce52ae","Type":"ContainerStarted","Data":"86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd"} Jul 03 00:00:01.186604 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.186575 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" event={"ID":"af0b0d68-206d-4d45-b0c3-aa11851d9aa7","Type":"ContainerStarted","Data":"4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243"} Jul 03 00:00:01.197547 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:01.197504 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-pruner-29717280-kqc4w" podStartSLOduration=1.197491654 podStartE2EDuration="1.197491654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:00:01.195378569 +0000 UTC m=+2255.279979987" watchObservedRunningTime="2026-07-03 00:00:01.197491654 +0000 UTC m=+2255.282093071" Jul 03 00:00:02.196474 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:02.196443 2562 generic.go:356] "Generic (PLEG): container finished" podID="7523c65e-d9d6-4bc0-b225-43c696b221e7" containerID="2a37cc02fef992007c4cb7232b93181b9c816a4c06ca17aa1babec966cbd8688" exitCode=0 Jul 03 00:00:02.196862 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:02.196532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29717280-kqc4w" event={"ID":"7523c65e-d9d6-4bc0-b225-43c696b221e7","Type":"ContainerDied","Data":"2a37cc02fef992007c4cb7232b93181b9c816a4c06ca17aa1babec966cbd8688"} Jul 03 00:00:03.346823 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.346793 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:03.536850 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.536819 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ks48\" (UniqueName: \"kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48\") pod \"7523c65e-d9d6-4bc0-b225-43c696b221e7\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " Jul 03 00:00:03.537022 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.536927 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca\") pod \"7523c65e-d9d6-4bc0-b225-43c696b221e7\" (UID: \"7523c65e-d9d6-4bc0-b225-43c696b221e7\") " Jul 03 00:00:03.537311 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.537277 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca" (OuterVolumeSpecName: "serviceca") pod "7523c65e-d9d6-4bc0-b225-43c696b221e7" (UID: "7523c65e-d9d6-4bc0-b225-43c696b221e7"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 00:00:03.539263 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.539240 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48" (OuterVolumeSpecName: "kube-api-access-4ks48") pod "7523c65e-d9d6-4bc0-b225-43c696b221e7" (UID: "7523c65e-d9d6-4bc0-b225-43c696b221e7"). InnerVolumeSpecName "kube-api-access-4ks48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:00:03.637896 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.637858 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ks48\" (UniqueName: \"kubernetes.io/projected/7523c65e-d9d6-4bc0-b225-43c696b221e7-kube-api-access-4ks48\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:00:03.637896 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:03.637891 2562 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7523c65e-d9d6-4bc0-b225-43c696b221e7-serviceca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:00:04.208990 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:04.208963 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29717280-kqc4w" Jul 03 00:00:04.209177 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:04.208960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29717280-kqc4w" event={"ID":"7523c65e-d9d6-4bc0-b225-43c696b221e7","Type":"ContainerDied","Data":"9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d"} Jul 03 00:00:04.209177 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:04.209084 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ac2674d9879340771c41915348aa458fccae2156ec5d3ae79a26ec44262849d" Jul 03 00:00:09.234886 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:09.234854 2562 generic.go:356] "Generic (PLEG): container finished" podID="26c1d823-bd8a-491b-8726-b569322be15c" containerID="792062e0f62572f6ff5d899882c40a40de407a5b9bc34c1b506f435b614df818" exitCode=0 Jul 03 00:00:09.235290 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:09.234907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" event={"ID":"26c1d823-bd8a-491b-8726-b569322be15c","Type":"ContainerDied","Data":"792062e0f62572f6ff5d899882c40a40de407a5b9bc34c1b506f435b614df818"} Jul 03 00:00:10.372297 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:10.372273 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:10.394883 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:10.394852 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq5nj\" (UniqueName: \"kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj\") pod \"26c1d823-bd8a-491b-8726-b569322be15c\" (UID: \"26c1d823-bd8a-491b-8726-b569322be15c\") " Jul 03 00:00:10.397120 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:10.397094 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj" (OuterVolumeSpecName: "kube-api-access-sq5nj") pod "26c1d823-bd8a-491b-8726-b569322be15c" (UID: "26c1d823-bd8a-491b-8726-b569322be15c"). InnerVolumeSpecName "kube-api-access-sq5nj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:00:10.496095 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:10.495999 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq5nj\" (UniqueName: \"kubernetes.io/projected/26c1d823-bd8a-491b-8726-b569322be15c-kube-api-access-sq5nj\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:00:11.244272 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:11.244240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" event={"ID":"26c1d823-bd8a-491b-8726-b569322be15c","Type":"ContainerDied","Data":"dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a"} Jul 03 00:00:11.244272 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:11.244271 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267" Jul 03 00:00:11.244482 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:11.244288 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcf149f18b3aacbb624dcc01facb26035db8c19b94afcfebfea0799b037f186a" Jul 03 00:00:29.326280 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:29.326241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" event={"ID":"af0b0d68-206d-4d45-b0c3-aa11851d9aa7","Type":"ContainerStarted","Data":"e282d5eafdae45fcd3944897f13f7ec9a1527de607553fbf24e93823d52f212a"} Jul 03 00:00:29.327819 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:29.327777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" event={"ID":"a195f5b6-96f5-4341-abec-2fecdfce52ae","Type":"ContainerStarted","Data":"de37b80da3979f5079b6ec67321aaf3a32b2a881e96e27aa3e46454a6f145876"} Jul 03 00:00:29.341657 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:29.341613 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" podStartSLOduration=1.728052893 podStartE2EDuration="29.341597775s" totalImagesPullingTime="27.613544882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:00:29.340130903 +0000 UTC m=+2283.424732336" watchObservedRunningTime="2026-07-03 00:00:29.341597775 +0000 UTC m=+2283.426199193" Jul 03 00:00:29.353445 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:29.353402 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" podStartSLOduration=1.535716004 podStartE2EDuration="29.353392296s" totalImagesPullingTime="27.817676292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:00:29.352861649 +0000 UTC m=+2283.437463068" watchObservedRunningTime="2026-07-03 00:00:29.353392296 +0000 UTC m=+2283.437993726" Jul 03 00:00:30.333446 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:30.333408 2562 generic.go:356] "Generic (PLEG): container finished" podID="a195f5b6-96f5-4341-abec-2fecdfce52ae" containerID="de37b80da3979f5079b6ec67321aaf3a32b2a881e96e27aa3e46454a6f145876" exitCode=0 Jul 03 00:00:30.333900 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:30.333498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" event={"ID":"a195f5b6-96f5-4341-abec-2fecdfce52ae","Type":"ContainerDied","Data":"de37b80da3979f5079b6ec67321aaf3a32b2a881e96e27aa3e46454a6f145876"} Jul 03 00:00:30.334831 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:30.334810 2562 generic.go:356] "Generic (PLEG): container finished" podID="af0b0d68-206d-4d45-b0c3-aa11851d9aa7" containerID="e282d5eafdae45fcd3944897f13f7ec9a1527de607553fbf24e93823d52f212a" exitCode=0 Jul 03 00:00:30.334904 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:30.334842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" event={"ID":"af0b0d68-206d-4d45-b0c3-aa11851d9aa7","Type":"ContainerDied","Data":"e282d5eafdae45fcd3944897f13f7ec9a1527de607553fbf24e93823d52f212a"} Jul 03 00:00:31.485631 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.485601 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:31.488846 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.488827 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:31.573637 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.573606 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsr84\" (UniqueName: \"kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84\") pod \"a195f5b6-96f5-4341-abec-2fecdfce52ae\" (UID: \"a195f5b6-96f5-4341-abec-2fecdfce52ae\") " Jul 03 00:00:31.573811 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.573658 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x28zq\" (UniqueName: \"kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq\") pod \"af0b0d68-206d-4d45-b0c3-aa11851d9aa7\" (UID: \"af0b0d68-206d-4d45-b0c3-aa11851d9aa7\") " Jul 03 00:00:31.575927 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.575889 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84" (OuterVolumeSpecName: "kube-api-access-fsr84") pod "a195f5b6-96f5-4341-abec-2fecdfce52ae" (UID: "a195f5b6-96f5-4341-abec-2fecdfce52ae"). InnerVolumeSpecName "kube-api-access-fsr84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:00:31.575927 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.575916 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq" (OuterVolumeSpecName: "kube-api-access-x28zq") pod "af0b0d68-206d-4d45-b0c3-aa11851d9aa7" (UID: "af0b0d68-206d-4d45-b0c3-aa11851d9aa7"). InnerVolumeSpecName "kube-api-access-x28zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:00:31.674657 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.674576 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsr84\" (UniqueName: \"kubernetes.io/projected/a195f5b6-96f5-4341-abec-2fecdfce52ae-kube-api-access-fsr84\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:00:31.674657 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:31.674605 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x28zq\" (UniqueName: \"kubernetes.io/projected/af0b0d68-206d-4d45-b0c3-aa11851d9aa7-kube-api-access-x28zq\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:00:32.345026 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.344970 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" Jul 03 00:00:32.345260 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.344978 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717280-x9f86" event={"ID":"a195f5b6-96f5-4341-abec-2fecdfce52ae","Type":"ContainerDied","Data":"86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd"} Jul 03 00:00:32.345260 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.345107 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86da77411b52bd9bb8bae353b565b9720b4cdc40c9696c5e29aa99a2e6eda5cd" Jul 03 00:00:32.346363 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.346346 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" Jul 03 00:00:32.346506 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.346345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888" event={"ID":"af0b0d68-206d-4d45-b0c3-aa11851d9aa7","Type":"ContainerDied","Data":"4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243"} Jul 03 00:00:32.346506 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:00:32.346472 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4914ddd48f72b960b54208ecbacfe45e203c76b98579ac03210f1829555fc243" Jul 03 00:02:54.368703 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.368618 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369140 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26c1d823-bd8a-491b-8726-b569322be15c" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369160 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26c1d823-bd8a-491b-8726-b569322be15c" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369181 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0b0d68-206d-4d45-b0c3-aa11851d9aa7" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369190 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0b0d68-206d-4d45-b0c3-aa11851d9aa7" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369198 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a195f5b6-96f5-4341-abec-2fecdfce52ae" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369207 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a195f5b6-96f5-4341-abec-2fecdfce52ae" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369219 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7523c65e-d9d6-4bc0-b225-43c696b221e7" containerName="image-pruner" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369227 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7523c65e-d9d6-4bc0-b225-43c696b221e7" containerName="image-pruner" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369314 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="af0b0d68-206d-4d45-b0c3-aa11851d9aa7" containerName="cleanup" Jul 03 00:02:54.369321 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369323 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a195f5b6-96f5-4341-abec-2fecdfce52ae" containerName="cleanup" Jul 03 00:02:54.369829 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369337 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7523c65e-d9d6-4bc0-b225-43c696b221e7" containerName="image-pruner" Jul 03 00:02:54.369829 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.369345 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26c1d823-bd8a-491b-8726-b569322be15c" containerName="cleanup" Jul 03 00:02:54.372474 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.372455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.374777 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.374747 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"openshift-service-ca.crt\"" Jul 03 00:02:54.376068 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.376046 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"sub-test-model-c1893b-kserve-self-signed-certs\"" Jul 03 00:02:54.376277 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.376081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"default-dockercfg-n8k7w\"" Jul 03 00:02:54.376454 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.376081 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1893b\"/\"kube-root-ca.crt\"" Jul 03 00:02:54.378895 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.378872 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:02:54.427483 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6n6\" (UniqueName: \"kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.427678 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.427678 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.427678 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.427894 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.427894 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.427749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528263 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528263 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528483 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6n6\" (UniqueName: \"kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528593 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528647 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.528703 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.529363 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.528762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.529593 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.529561 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.529781 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.529757 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.532687 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.532662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.534098 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.534064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.536622 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.536591 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6n6\" (UniqueName: \"kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6\") pod \"sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.681718 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.681629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:02:54.813588 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.813561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:02:54.815285 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:02:54.815251 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc724a0d0_bc47_4393_85a5_4b2344be5d72.slice/crio-7e3220a7ad751692bb9a4f1dd84438db972480e84d26ee666dc7ffbfa882de98 WatchSource:0}: Error finding container 7e3220a7ad751692bb9a4f1dd84438db972480e84d26ee666dc7ffbfa882de98: Status 404 returned error can't find the container with id 7e3220a7ad751692bb9a4f1dd84438db972480e84d26ee666dc7ffbfa882de98 Jul 03 00:02:54.984356 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.984323 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" event={"ID":"c724a0d0-bc47-4393-85a5-4b2344be5d72","Type":"ContainerStarted","Data":"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572"} Jul 03 00:02:54.984534 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:54.984364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" event={"ID":"c724a0d0-bc47-4393-85a5-4b2344be5d72","Type":"ContainerStarted","Data":"7e3220a7ad751692bb9a4f1dd84438db972480e84d26ee666dc7ffbfa882de98"} Jul 03 00:02:55.000133 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:02:55.000086 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" podStartSLOduration=1.000067433 podStartE2EDuration="1.000067433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 00:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:02:54.999116382 +0000 UTC m=+2429.083717799" watchObservedRunningTime="2026-07-03 00:02:55.000067433 +0000 UTC m=+2429.084668852" Jul 03 00:03:04.682483 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:03:04.682431 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:03:04.682483 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:03:04.682499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:03:04.695406 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:03:04.695378 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:03:05.040622 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:03:05.040591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:04:24.490389 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:24.490307 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:04:39.513140 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.513097 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" podUID="c724a0d0-bc47-4393-85a5-4b2344be5d72" containerName="main" containerID="cri-o://18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572" gracePeriod=45 Jul 03 00:04:39.775851 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.775787 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:04:39.878225 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878195 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878381 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878285 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878381 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878381 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878338 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878381 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878361 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878594 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878383 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj6n6\" (UniqueName: \"kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6\") pod \"c724a0d0-bc47-4393-85a5-4b2344be5d72\" (UID: \"c724a0d0-bc47-4393-85a5-4b2344be5d72\") " Jul 03 00:04:39.878594 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878479 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache" (OuterVolumeSpecName: "model-cache") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:39.878688 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878588 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:39.878688 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878620 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home" (OuterVolumeSpecName: "home") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:39.878767 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878703 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-tmp-dir\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:39.878767 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.878719 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-model-cache\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:39.880684 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.880662 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:04:39.880684 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.880659 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6" (OuterVolumeSpecName: "kube-api-access-fj6n6") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "kube-api-access-fj6n6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:04:39.880684 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.880656 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm" (OuterVolumeSpecName: "dshm") pod "c724a0d0-bc47-4393-85a5-4b2344be5d72" (UID: "c724a0d0-bc47-4393-85a5-4b2344be5d72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 00:04:39.979976 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.979939 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-dshm\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:39.979976 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.979969 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c724a0d0-bc47-4393-85a5-4b2344be5d72-tls-certs\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:39.979976 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.979981 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c724a0d0-bc47-4393-85a5-4b2344be5d72-home\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:39.980230 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:39.979990 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj6n6\" (UniqueName: \"kubernetes.io/projected/c724a0d0-bc47-4393-85a5-4b2344be5d72-kube-api-access-fj6n6\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:04:40.465486 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.465456 2562 generic.go:356] "Generic (PLEG): container finished" podID="c724a0d0-bc47-4393-85a5-4b2344be5d72" containerID="18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572" exitCode=0 Jul 03 00:04:40.465671 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.465494 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" Jul 03 00:04:40.465671 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.465502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" event={"ID":"c724a0d0-bc47-4393-85a5-4b2344be5d72","Type":"ContainerDied","Data":"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572"} Jul 03 00:04:40.465671 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.465541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh" event={"ID":"c724a0d0-bc47-4393-85a5-4b2344be5d72","Type":"ContainerDied","Data":"7e3220a7ad751692bb9a4f1dd84438db972480e84d26ee666dc7ffbfa882de98"} Jul 03 00:04:40.465671 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.465564 2562 scope.go:117] "RemoveContainer" containerID="18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572" Jul 03 00:04:40.474115 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.474096 2562 scope.go:117] "RemoveContainer" containerID="18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572" Jul 03 00:04:40.474365 ip-10-0-130-160 kubenswrapper[2562]: E0703 00:04:40.474344 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572\": container with ID starting with 18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572 not found: ID does not exist" containerID="18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572" Jul 03 00:04:40.474406 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.474375 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572"} err="failed to get container status \"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572\": rpc error: code = NotFound desc = could not find container \"18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572\": container with ID starting with 18063caced8f026fa6a334f7a8f8c749ca09cd5c59360f578bfd84db41b79572 not found: ID does not exist" Jul 03 00:04:40.480810 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.480789 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:04:40.485229 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:40.485203 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1893b/sub-test-model-c1893b-kserve-5d6b9d5c6c-8twnh"] Jul 03 00:04:42.432742 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:04:42.432707 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c724a0d0-bc47-4393-85a5-4b2344be5d72" path="/var/lib/kubelet/pods/c724a0d0-bc47-4393-85a5-4b2344be5d72/volumes" Jul 03 00:05:42.083605 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.083567 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 03 00:05:42.084107 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.083777 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" podUID="533ca191-7818-4a22-a9fb-d7deae542b7e" containerName="maas-api" containerID="cri-o://706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9" gracePeriod=30 Jul 03 00:05:42.116049 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.115992 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888"] Jul 03 00:05:42.116234 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.116073 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-c1893b-29717280-z5888"] Jul 03 00:05:42.336293 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.336215 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 03 00:05:42.433857 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.433829 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0b0d68-206d-4d45-b0c3-aa11851d9aa7" path="/var/lib/kubelet/pods/af0b0d68-206d-4d45-b0c3-aa11851d9aa7/volumes" Jul 03 00:05:42.434145 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.434127 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca\") pod \"533ca191-7818-4a22-a9fb-d7deae542b7e\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " Jul 03 00:05:42.434224 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.434196 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") pod \"533ca191-7818-4a22-a9fb-d7deae542b7e\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " Jul 03 00:05:42.434285 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.434258 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwhb4\" (UniqueName: \"kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4\") pod \"533ca191-7818-4a22-a9fb-d7deae542b7e\" (UID: \"533ca191-7818-4a22-a9fb-d7deae542b7e\") " Jul 03 00:05:42.434563 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.434537 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "533ca191-7818-4a22-a9fb-d7deae542b7e" (UID: "533ca191-7818-4a22-a9fb-d7deae542b7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 00:05:42.436429 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.436404 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4" (OuterVolumeSpecName: "kube-api-access-xwhb4") pod "533ca191-7818-4a22-a9fb-d7deae542b7e" (UID: "533ca191-7818-4a22-a9fb-d7deae542b7e"). InnerVolumeSpecName "kube-api-access-xwhb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:05:42.436429 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.436416 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "533ca191-7818-4a22-a9fb-d7deae542b7e" (UID: "533ca191-7818-4a22-a9fb-d7deae542b7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:05:42.535732 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.535685 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwhb4\" (UniqueName: \"kubernetes.io/projected/533ca191-7818-4a22-a9fb-d7deae542b7e-kube-api-access-xwhb4\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:05:42.535732 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.535724 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ca191-7818-4a22-a9fb-d7deae542b7e-openshift-service-ca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:05:42.535732 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.535740 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/533ca191-7818-4a22-a9fb-d7deae542b7e-maas-api-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:05:42.755202 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.755165 2562 generic.go:356] "Generic (PLEG): container finished" podID="533ca191-7818-4a22-a9fb-d7deae542b7e" containerID="706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9" exitCode=0 Jul 03 00:05:42.755375 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.755208 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" Jul 03 00:05:42.755375 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.755231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" event={"ID":"533ca191-7818-4a22-a9fb-d7deae542b7e","Type":"ContainerDied","Data":"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9"} Jul 03 00:05:42.755375 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.755275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s" event={"ID":"533ca191-7818-4a22-a9fb-d7deae542b7e","Type":"ContainerDied","Data":"f33a71f9eb5c8e04f637f7e36cc81b00cee97d76dfda498079af675690a3f21f"} Jul 03 00:05:42.755375 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.755298 2562 scope.go:117] "RemoveContainer" containerID="706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9" Jul 03 00:05:42.765373 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.765356 2562 scope.go:117] "RemoveContainer" containerID="706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9" Jul 03 00:05:42.765604 ip-10-0-130-160 kubenswrapper[2562]: E0703 00:05:42.765582 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9\": container with ID starting with 706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9 not found: ID does not exist" containerID="706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9" Jul 03 00:05:42.765665 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.765613 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9"} err="failed to get container status \"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9\": rpc error: code = NotFound desc = could not find container \"706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9\": container with ID starting with 706de82f5a1a9e928895d13c3c7fac6fa33d6de1c7b782fd6cf560f3a0cd10d9 not found: ID does not exist" Jul 03 00:05:42.775973 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.775949 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 03 00:05:42.780185 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:42.780162 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c1893b-68f94f8d64-kt84s"] Jul 03 00:05:44.431986 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:05:44.431956 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533ca191-7818-4a22-a9fb-d7deae542b7e" path="/var/lib/kubelet/pods/533ca191-7818-4a22-a9fb-d7deae542b7e/volumes" Jul 03 00:06:25.243901 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.243866 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 03 00:06:25.244506 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.244143 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" podUID="e8553366-6e6b-4c9c-841d-a27560e73ce5" containerName="maas-api" containerID="cri-o://f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b" gracePeriod=30 Jul 03 00:06:25.270411 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.270383 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267"] Jul 03 00:06:25.271953 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.271933 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-c04a64-29717280-kj267"] Jul 03 00:06:25.503225 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.503154 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 03 00:06:25.612877 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.612827 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") pod \"e8553366-6e6b-4c9c-841d-a27560e73ce5\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " Jul 03 00:06:25.613091 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.612932 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca\") pod \"e8553366-6e6b-4c9c-841d-a27560e73ce5\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " Jul 03 00:06:25.613091 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.612955 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj824\" (UniqueName: \"kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824\") pod \"e8553366-6e6b-4c9c-841d-a27560e73ce5\" (UID: \"e8553366-6e6b-4c9c-841d-a27560e73ce5\") " Jul 03 00:06:25.613313 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.613278 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8553366-6e6b-4c9c-841d-a27560e73ce5" (UID: "e8553366-6e6b-4c9c-841d-a27560e73ce5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 00:06:25.615172 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.615144 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8553366-6e6b-4c9c-841d-a27560e73ce5" (UID: "e8553366-6e6b-4c9c-841d-a27560e73ce5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 00:06:25.615283 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.615186 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824" (OuterVolumeSpecName: "kube-api-access-gj824") pod "e8553366-6e6b-4c9c-841d-a27560e73ce5" (UID: "e8553366-6e6b-4c9c-841d-a27560e73ce5"). InnerVolumeSpecName "kube-api-access-gj824". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 00:06:25.714181 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.714126 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8553366-6e6b-4c9c-841d-a27560e73ce5-openshift-service-ca\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:06:25.714181 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.714173 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj824\" (UniqueName: \"kubernetes.io/projected/e8553366-6e6b-4c9c-841d-a27560e73ce5-kube-api-access-gj824\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:06:25.714181 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.714185 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8553366-6e6b-4c9c-841d-a27560e73ce5-maas-api-tls\") on node \"ip-10-0-130-160.ec2.internal\" DevicePath \"\"" Jul 03 00:06:25.950949 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.950864 2562 generic.go:356] "Generic (PLEG): container finished" podID="e8553366-6e6b-4c9c-841d-a27560e73ce5" containerID="f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b" exitCode=0 Jul 03 00:06:25.950949 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.950903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" event={"ID":"e8553366-6e6b-4c9c-841d-a27560e73ce5","Type":"ContainerDied","Data":"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b"} Jul 03 00:06:25.950949 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.950915 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" Jul 03 00:06:25.950949 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.950947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd" event={"ID":"e8553366-6e6b-4c9c-841d-a27560e73ce5","Type":"ContainerDied","Data":"43fe92741709c3f757062b0bc8def51bc6186d11fbfb404878dc04790e9fe090"} Jul 03 00:06:25.951280 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.950970 2562 scope.go:117] "RemoveContainer" containerID="f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b" Jul 03 00:06:25.967973 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.967948 2562 scope.go:117] "RemoveContainer" containerID="f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b" Jul 03 00:06:25.968330 ip-10-0-130-160 kubenswrapper[2562]: E0703 00:06:25.968305 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b\": container with ID starting with f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b not found: ID does not exist" containerID="f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b" Jul 03 00:06:25.968409 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.968342 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b"} err="failed to get container status \"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b\": rpc error: code = NotFound desc = could not find container \"f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b\": container with ID starting with f07468af2055311e2623b832b4101728dec97c991738d62cff8ff4b65db0775b not found: ID does not exist" Jul 03 00:06:25.977138 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.977107 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 03 00:06:25.980848 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:25.980814 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c04a64-6df8dfc845-gnbfd"] Jul 03 00:06:26.432455 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:26.432419 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26c1d823-bd8a-491b-8726-b569322be15c" path="/var/lib/kubelet/pods/26c1d823-bd8a-491b-8726-b569322be15c/volumes" Jul 03 00:06:26.432872 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:26.432856 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8553366-6e6b-4c9c-841d-a27560e73ce5" path="/var/lib/kubelet/pods/e8553366-6e6b-4c9c-841d-a27560e73ce5/volumes" Jul 03 00:06:26.617648 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:26.617619 2562 scope.go:117] "RemoveContainer" containerID="792062e0f62572f6ff5d899882c40a40de407a5b9bc34c1b506f435b614df818" Jul 03 00:06:31.803209 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:31.803175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-99ffcd69c-llw4h_47c106ae-ac92-4b1e-953f-3d9e6ef08071/authorino/0.log" Jul 03 00:06:36.579125 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:36.579093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hnrnj_a49512f8-964c-4134-be9d-f73f783a056d/manager/0.log" Jul 03 00:06:36.719665 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:36.719632 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5zpnr_60eb6602-3774-40b3-a041-ed2ad78431f2/manager/0.log" Jul 03 00:06:37.092579 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:37.092546 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717280-x9f86_a195f5b6-96f5-4341-abec-2fecdfce52ae/cleanup/0.log" Jul 03 00:06:37.588618 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:37.588580 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-9dl8g_b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b/manager/0.log" Jul 03 00:06:37.594675 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:37.594653 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-9dl8g_b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b/copy-manifests/0.log" Jul 03 00:06:39.330443 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:39.330406 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-99ffcd69c-llw4h_47c106ae-ac92-4b1e-953f-3d9e6ef08071/authorino/0.log" Jul 03 00:06:39.455986 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:39.455951 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-87tdv_2fb7115d-336b-4633-8dfc-c818097bfcef/manager/0.log" Jul 03 00:06:40.533582 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:40.533549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787fvrx_96b7afa5-a99f-41ab-be30-31df7e653bcf/istio-proxy/0.log" Jul 03 00:06:41.010652 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:41.010606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/1.log" Jul 03 00:06:41.267546 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:41.267472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-28sgd_5664fc42-a2cf-4b95-84c3-dee1f3aa6289/payload-processing/0.log" Jul 03 00:06:41.746808 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:41.746778 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-plznc_8bd092f0-05c5-42a5-837b-a5140dad7463/main/0.log" Jul 03 00:06:41.993763 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:41.993719 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gmtnc_56a66842-1c80-4232-8ef0-38e3f7014839/main/0.log" Jul 03 00:06:50.161949 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:50.161918 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q5gzx_694a6e24-4662-4aca-b3f8-bc4e9e3ae9ba/global-pull-secret-syncer/0.log" Jul 03 00:06:50.280128 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:50.280093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ghrxx_c4ada550-0bb9-4c82-bf50-e9cf09dd0713/konnectivity-agent/0.log" Jul 03 00:06:50.367540 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:50.367507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-160.ec2.internal_a5e8112810e8b8f02dbe6afdacc66d20/haproxy/0.log" Jul 03 00:06:55.877926 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:55.877881 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-99ffcd69c-llw4h_47c106ae-ac92-4b1e-953f-3d9e6ef08071/authorino/0.log" Jul 03 00:06:55.910553 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:55.910514 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-87tdv_2fb7115d-336b-4633-8dfc-c818097bfcef/manager/0.log" Jul 03 00:06:58.183149 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.183107 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bqghz_2f9aec65-bd86-4e1a-913c-032fcc6e6cab/monitoring-plugin/0.log" Jul 03 00:06:58.366782 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.366745 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qqhql_e6675f38-fa06-4f58-bd0b-3934250ded1b/node-exporter/0.log" Jul 03 00:06:58.384704 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.384678 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qqhql_e6675f38-fa06-4f58-bd0b-3934250ded1b/kube-rbac-proxy/0.log" Jul 03 00:06:58.403036 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.402989 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qqhql_e6675f38-fa06-4f58-bd0b-3934250ded1b/init-textfile/0.log" Jul 03 00:06:58.502543 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.502518 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/prometheus/0.log" Jul 03 00:06:58.520384 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.520357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/config-reloader/0.log" Jul 03 00:06:58.540170 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.540140 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/thanos-sidecar/0.log" Jul 03 00:06:58.558235 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.558209 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/kube-rbac-proxy-web/0.log" Jul 03 00:06:58.576847 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.576818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/kube-rbac-proxy/0.log" Jul 03 00:06:58.595210 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.595185 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/kube-rbac-proxy-thanos/0.log" Jul 03 00:06:58.616723 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:58.616699 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_16e92d21-14c8-42c1-bf50-e5177c600bfa/init-config-reloader/0.log" Jul 03 00:06:59.876956 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.876926 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f"] Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877257 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c724a0d0-bc47-4393-85a5-4b2344be5d72" containerName="main" Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877268 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c724a0d0-bc47-4393-85a5-4b2344be5d72" containerName="main" Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877285 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8553366-6e6b-4c9c-841d-a27560e73ce5" containerName="maas-api" Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877290 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8553366-6e6b-4c9c-841d-a27560e73ce5" containerName="maas-api" Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877299 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533ca191-7818-4a22-a9fb-d7deae542b7e" containerName="maas-api" Jul 03 00:06:59.877328 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877304 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="533ca191-7818-4a22-a9fb-d7deae542b7e" containerName="maas-api" Jul 03 00:06:59.877588 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877357 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8553366-6e6b-4c9c-841d-a27560e73ce5" containerName="maas-api" Jul 03 00:06:59.877588 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877366 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="533ca191-7818-4a22-a9fb-d7deae542b7e" containerName="maas-api" Jul 03 00:06:59.877588 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.877371 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c724a0d0-bc47-4393-85a5-4b2344be5d72" containerName="main" Jul 03 00:06:59.880558 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.880539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.882834 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.882812 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x5drt\"/\"kube-root-ca.crt\"" Jul 03 00:06:59.882944 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.882871 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x5drt\"/\"openshift-service-ca.crt\"" Jul 03 00:06:59.882944 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.882900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x5drt\"/\"default-dockercfg-cjp6x\"" Jul 03 00:06:59.890308 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.890287 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f"] Jul 03 00:06:59.912420 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.912391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-proc\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.912420 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.912419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-lib-modules\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.912578 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.912503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-sys\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.912578 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.912532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-podres\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.912578 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.912551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhg9j\" (UniqueName: \"kubernetes.io/projected/969db373-604f-44c4-85a3-2db9dfa4883d-kube-api-access-qhg9j\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:06:59.986128 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:06:59.986096 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-ckp6s_203ac940-1810-4a96-b963-d6145fc9de78/networking-console-plugin/0.log" Jul 03 00:07:00.013280 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-proc\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013280 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-lib-modules\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013452 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-sys\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013452 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-podres\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013452 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhg9j\" (UniqueName: \"kubernetes.io/projected/969db373-604f-44c4-85a3-2db9dfa4883d-kube-api-access-qhg9j\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013452 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013371 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-proc\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013452 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-sys\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013617 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013492 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-lib-modules\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.013617 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.013497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/969db373-604f-44c4-85a3-2db9dfa4883d-podres\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.021023 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.020979 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhg9j\" (UniqueName: \"kubernetes.io/projected/969db373-604f-44c4-85a3-2db9dfa4883d-kube-api-access-qhg9j\") pod \"perf-node-gather-daemonset-zjh8f\" (UID: \"969db373-604f-44c4-85a3-2db9dfa4883d\") " pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.190734 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.190644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:00.322345 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:00.322320 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f"] Jul 03 00:07:00.324254 ip-10-0-130-160 kubenswrapper[2562]: W0703 00:07:00.324217 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod969db373_604f_44c4_85a3_2db9dfa4883d.slice/crio-ce9f0053be261c48e268e1917a9398926f7fbae21d519ca820937c61dd12004e WatchSource:0}: Error finding container ce9f0053be261c48e268e1917a9398926f7fbae21d519ca820937c61dd12004e: Status 404 returned error can't find the container with id ce9f0053be261c48e268e1917a9398926f7fbae21d519ca820937c61dd12004e Jul 03 00:07:01.109226 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:01.109192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" event={"ID":"969db373-604f-44c4-85a3-2db9dfa4883d","Type":"ContainerStarted","Data":"be28792f1d704183ef963548aca58e490be3c47f0aa90cf032a70a3ce312d5db"} Jul 03 00:07:01.109647 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:01.109242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" event={"ID":"969db373-604f-44c4-85a3-2db9dfa4883d","Type":"ContainerStarted","Data":"ce9f0053be261c48e268e1917a9398926f7fbae21d519ca820937c61dd12004e"} Jul 03 00:07:01.109647 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:01.109321 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:01.124362 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:01.124316 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" podStartSLOduration=2.124278216 podStartE2EDuration="2.124278216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 00:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 00:07:01.122641005 +0000 UTC m=+2675.207242424" watchObservedRunningTime="2026-07-03 00:07:01.124278216 +0000 UTC m=+2675.208879633" Jul 03 00:07:02.488750 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:02.488722 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ckbs_a657f196-64df-4605-9d85-e726752bd25a/dns/0.log" Jul 03 00:07:02.508264 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:02.508229 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ckbs_a657f196-64df-4605-9d85-e726752bd25a/kube-rbac-proxy/0.log" Jul 03 00:07:02.611737 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:02.611710 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hxd76_69b0b64a-23c1-45fb-84ba-9abd7c29442f/dns-node-resolver/0.log" Jul 03 00:07:03.127379 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:03.127351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29717280-kqc4w_7523c65e-d9d6-4bc0-b225-43c696b221e7/image-pruner/0.log" Jul 03 00:07:03.241488 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:03.241462 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r2fwr_cdd68c25-ded4-49cb-bf2a-d85ac74f44d3/node-ca/0.log" Jul 03 00:07:04.117235 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:04.117203 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787fvrx_96b7afa5-a99f-41ab-be30-31df7e653bcf/istio-proxy/0.log" Jul 03 00:07:04.291250 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:04.291219 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/0.log" Jul 03 00:07:04.348678 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:04.348646 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/1.log" Jul 03 00:07:04.450495 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:04.450404 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-28sgd_5664fc42-a2cf-4b95-84c3-dee1f3aa6289/payload-processing/0.log" Jul 03 00:07:05.178860 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:05.178816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtxgv_cf2d2948-d60c-4fb4-8fed-aa89cf5b3d8f/serve-healthcheck-canary/0.log" Jul 03 00:07:05.624485 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:05.624457 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cdcm_c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a/kube-rbac-proxy/0.log" Jul 03 00:07:05.642942 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:05.642920 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cdcm_c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a/exporter/0.log" Jul 03 00:07:05.662864 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:05.662838 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cdcm_c751b8b4-0cb8-4bc7-bc38-8cf4c722d02a/extractor/0.log" Jul 03 00:07:07.124185 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:07.124146 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x5drt/perf-node-gather-daemonset-zjh8f" Jul 03 00:07:07.772509 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:07.772481 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hnrnj_a49512f8-964c-4134-be9d-f73f783a056d/manager/0.log" Jul 03 00:07:07.983640 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:07.983613 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5zpnr_60eb6602-3774-40b3-a041-ed2ad78431f2/manager/0.log" Jul 03 00:07:08.062115 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:08.062037 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717280-x9f86_a195f5b6-96f5-4341-abec-2fecdfce52ae/cleanup/0.log" Jul 03 00:07:08.275023 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:08.274978 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-9dl8g_b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b/manager/0.log" Jul 03 00:07:08.292800 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:08.292778 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f45646c76-9dl8g_b35eb8ad-dac0-4ed4-a9d2-881ad7566d7b/copy-manifests/0.log" Jul 03 00:07:09.906377 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:09.906296 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787fvrx_96b7afa5-a99f-41ab-be30-31df7e653bcf/istio-proxy/0.log" Jul 03 00:07:09.982078 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:09.982045 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-679bd8f54d-hv7kx_59a24c20-0014-419c-a5dc-672abbb024da/manager/0.log" Jul 03 00:07:10.046161 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:10.046126 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/0.log" Jul 03 00:07:10.099198 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:10.099165 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/1.log" Jul 03 00:07:10.191475 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:10.191360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-28sgd_5664fc42-a2cf-4b95-84c3-dee1f3aa6289/payload-processing/0.log" Jul 03 00:07:11.376261 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:11.376229 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6787fvrx_96b7afa5-a99f-41ab-be30-31df7e653bcf/istio-proxy/0.log" Jul 03 00:07:11.517693 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:11.517666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/0.log" Jul 03 00:07:11.575903 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:11.575812 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2zgmk_487b9ec5-8615-44ec-95cf-699ec5edbde5/istio-proxy/1.log" Jul 03 00:07:11.661155 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:11.661125 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-28sgd_5664fc42-a2cf-4b95-84c3-dee1f3aa6289/payload-processing/0.log" Jul 03 00:07:15.915980 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:15.915943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7jnk5_0c371128-dafc-4f47-a55c-7ae0e7ceeced/kube-multus/0.log" Jul 03 00:07:15.937104 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:15.937078 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/kube-multus-additional-cni-plugins/0.log" Jul 03 00:07:15.956140 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:15.956104 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/egress-router-binary-copy/0.log" Jul 03 00:07:15.974650 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:15.974618 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/cni-plugins/0.log" Jul 03 00:07:15.992966 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:15.992937 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/bond-cni-plugin/0.log" Jul 03 00:07:16.010921 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:16.010893 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/routeoverride-cni/0.log" Jul 03 00:07:16.029148 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:16.029083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/whereabouts-cni-bincopy/0.log" Jul 03 00:07:16.047776 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:16.047748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k94b5_ed8fda13-208f-47f1-bba0-e3f41fb34716/whereabouts-cni/0.log" Jul 03 00:07:16.489419 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:16.489341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wq5cf_795d14f0-d610-4fb2-9ccc-adc4f3478766/network-metrics-daemon/0.log" Jul 03 00:07:16.504936 ip-10-0-130-160 kubenswrapper[2562]: I0703 00:07:16.504909 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wq5cf_795d14f0-d610-4fb2-9ccc-adc4f3478766/kube-rbac-proxy/0.log"