Jul 09 14:54:26.991157 ip-10-0-142-231 systemd[1]: Starting Kubernetes Kubelet... Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.455916 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461903 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461916 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461920 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461923 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461926 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461929 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461932 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:27.473403 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461935 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461938 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461941 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461943 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461946 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461948 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461951 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461954 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461957 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461960 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461963 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461965 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461968 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461971 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461977 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461980 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461982 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461985 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461988 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:27.476482 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461990 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461992 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.461996 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462000 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462003 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462005 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462008 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462011 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462015 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462019 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462022 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462024 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462026 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462029 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462032 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462035 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462037 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462040 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462043 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:27.564335 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462046 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:27.530640 ip-10-0-142-231 systemd[1]: Started Kubernetes Kubelet. Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462049 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462051 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462054 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462056 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462058 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462061 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462063 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462066 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462069 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462071 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462074 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462076 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462078 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462081 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462083 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462086 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462089 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462092 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462094 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:27.609093 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462097 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462099 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462103 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462105 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462108 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462110 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462112 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462115 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462117 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462119 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462123 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462125 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462128 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462130 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462133 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462135 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462138 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462140 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462143 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462145 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:27.814266 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462148 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462151 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462154 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462157 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462160 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462163 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462165 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462168 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462171 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462174 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462177 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462179 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462182 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462184 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462864 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462870 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462888 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462891 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462894 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:27.815246 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462898 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462902 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462905 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462908 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462911 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462914 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462917 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462919 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462922 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462926 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462929 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462932 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462935 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462938 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462941 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462944 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462947 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462951 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462953 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462956 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:27.816239 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462958 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462961 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462963 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462966 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462968 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462971 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462973 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462975 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462978 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462980 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462983 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462985 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462987 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462990 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462993 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462996 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.462998 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463001 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463004 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463006 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:27.817040 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463009 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463011 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463014 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463016 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463019 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463021 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463024 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463026 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463029 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463031 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463034 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463036 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463039 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463041 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463043 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463046 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463048 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463054 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463057 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463059 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:27.817609 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463062 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463064 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463067 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463070 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463072 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463074 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463077 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463080 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463082 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463086 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463088 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463091 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463093 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463096 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463098 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463101 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463103 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463106 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463108 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:27.818288 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463111 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463113 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463115 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463118 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463121 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463123 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463126 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463128 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463130 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463133 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463137 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463139 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463142 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463144 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463146 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463493 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463498 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463501 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463503 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:27.818820 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463506 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463509 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463511 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463514 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463516 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463519 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463523 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463525 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463528 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463530 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463533 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463537 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463541 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463543 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463547 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463550 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463553 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463556 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463559 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:27.819414 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463564 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463567 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463569 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463571 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463576 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463578 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463581 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463583 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463586 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463588 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463591 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463593 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463595 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463598 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463600 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463603 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463605 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463607 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463610 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463612 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:27.819962 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463614 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463617 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463619 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463621 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463624 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463626 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463628 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463630 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463633 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463636 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463640 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463642 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463645 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463648 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463650 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463653 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463656 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463659 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463662 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463664 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:27.820564 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463667 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463669 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463671 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463674 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463676 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463678 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463681 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463683 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463686 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463688 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463690 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463693 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463695 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463698 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463700 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463703 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463705 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463707 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463710 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463712 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:27.821152 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463714 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463719 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463722 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463725 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463729 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463732 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463734 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463736 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463740 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463743 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463745 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463747 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463750 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463752 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463755 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.463757 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463823 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463830 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463836 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463841 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 14:54:27.821786 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463844 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463848 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463852 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463857 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463860 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463863 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463866 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463869 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463884 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463887 2558 flags.go:64] FLAG: --cgroup-root="" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463891 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463894 2558 flags.go:64] FLAG: --client-ca-file="" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463897 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463900 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463904 2558 flags.go:64] FLAG: --cluster-domain="" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463907 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463911 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463915 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463918 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463922 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463926 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463929 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463933 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463936 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 09 14:54:27.822379 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463939 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463942 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463945 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463947 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463951 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463954 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463957 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463959 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463962 2558 flags.go:64] FLAG: --enable-server="true" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.463965 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464039 2558 flags.go:64] FLAG: --event-burst="100" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464044 2558 flags.go:64] FLAG: --event-qps="50" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464049 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464054 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464132 2558 flags.go:64] FLAG: --eviction-hard="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464160 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464168 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464175 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464183 2558 flags.go:64] FLAG: --eviction-soft="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464188 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464194 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464199 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464211 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464216 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464221 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 14:54:27.823085 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464226 2558 flags.go:64] FLAG: --feature-gates="" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464232 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464238 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464243 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464249 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464254 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464264 2558 flags.go:64] FLAG: --help="false" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464270 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-142-231.ec2.internal" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464275 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464281 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464286 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464292 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464300 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464305 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464315 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464320 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464325 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464330 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464336 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464341 2558 flags.go:64] FLAG: --kube-reserved="" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464346 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464351 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464357 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464367 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 14:54:27.823736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464371 2558 flags.go:64] FLAG: --lock-file="" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464376 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464381 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464386 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464395 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464401 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464405 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464415 2558 flags.go:64] FLAG: --logging-format="text" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464420 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464425 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464430 2558 flags.go:64] FLAG: --manifest-url="" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464435 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464442 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464447 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464454 2558 flags.go:64] FLAG: --max-pods="110" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464464 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464469 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464473 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464479 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464484 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464489 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464495 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464512 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464519 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464534 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 14:54:27.824405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464539 2558 flags.go:64] FLAG: --pod-cidr="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464544 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464552 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464557 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464561 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464570 2558 flags.go:64] FLAG: --port="10250" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464580 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464585 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a0c62c2f6ec55b41" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464592 2558 flags.go:64] FLAG: --qos-reserved="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464597 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464602 2558 flags.go:64] FLAG: --register-node="true" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464606 2558 flags.go:64] FLAG: --register-with-taints="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464613 2558 flags.go:64] FLAG: --registry-burst="10" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464618 2558 flags.go:64] FLAG: --registry-qps="5" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464623 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464632 2558 flags.go:64] FLAG: --reserved-memory="" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464638 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464643 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464647 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464652 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464656 2558 flags.go:64] FLAG: --runonce="false" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464661 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464666 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464676 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464681 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 14:54:27.825104 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464686 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464691 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464696 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464701 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464705 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464709 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464714 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464722 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464727 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464732 2558 flags.go:64] FLAG: --system-cgroups="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464736 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464747 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464755 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464787 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464833 2558 flags.go:64] FLAG: --tls-min-version="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464852 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464857 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464861 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464866 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464870 2558 flags.go:64] FLAG: --v="2" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464921 2558 flags.go:64] FLAG: --version="false" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464926 2558 flags.go:64] FLAG: --vmodule="" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464931 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.464935 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 14:54:27.825767 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465153 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465160 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465164 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465168 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465171 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465174 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465176 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465179 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465185 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465187 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465190 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465196 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465199 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465202 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465205 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465207 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465210 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465213 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465216 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465219 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:27.826390 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465224 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465227 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465230 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465234 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465237 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465240 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465243 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465246 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465253 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465257 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465260 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465262 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465267 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465270 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465273 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465276 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465278 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465281 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465300 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465408 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:27.826969 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465411 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465415 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465417 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465421 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465425 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465428 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465430 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465433 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465436 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465438 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465441 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465443 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465446 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465450 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465454 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465457 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465460 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465463 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465466 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:27.827763 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465469 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465472 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465474 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465477 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465479 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465482 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465484 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465487 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465489 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465492 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465495 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465498 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465500 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465503 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465505 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465507 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465511 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465515 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465518 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465520 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:27.828309 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465523 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465526 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465529 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465531 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465534 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465537 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465539 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465542 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465544 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465547 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465549 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465552 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465554 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465557 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465559 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465562 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465564 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465567 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465570 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:28.085256 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.465572 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.465581 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.472406 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.472421 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.473366 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.473380 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473456 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473464 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473468 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473473 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473477 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473481 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473484 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473488 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473492 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:28.086530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473496 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473500 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473503 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473508 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473512 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473516 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473519 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473524 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473527 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473531 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473535 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473540 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473544 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473547 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473551 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473555 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473558 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473562 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473566 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:28.087173 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473569 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473574 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473578 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473582 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473585 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473589 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473593 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473597 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473601 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473605 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473611 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473617 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473621 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473625 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473629 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473633 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473636 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473640 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473644 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:28.087821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473647 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473652 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473656 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473660 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473663 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473667 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473671 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473675 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473679 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473683 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473689 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473695 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473700 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473705 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473709 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473713 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473718 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473722 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473726 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473730 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:28.088488 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473734 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473739 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473743 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473747 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473751 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473756 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473759 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473764 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473768 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473773 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473777 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473782 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473786 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473790 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473802 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473807 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473811 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473816 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473820 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473826 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:28.089149 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473831 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473835 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473839 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473842 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473846 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473851 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473855 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473859 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473863 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473867 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473891 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.473896 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.473904 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474088 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474096 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:54:28.090235 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474103 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474108 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474112 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474116 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474120 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474124 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474129 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474133 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474137 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474141 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474145 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474149 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474155 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474160 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474164 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474171 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474177 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474183 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474187 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474192 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:54:28.090866 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474196 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474200 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474204 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474208 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474212 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474216 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474220 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474224 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474227 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474231 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474235 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474239 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474245 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474249 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474253 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474257 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474261 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474265 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474269 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474273 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474278 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:54:28.091484 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474281 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474285 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474290 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474296 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474301 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474305 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474309 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474313 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474317 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474322 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474327 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474331 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474335 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474339 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474343 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474347 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474351 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474356 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474359 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:54:28.092302 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474364 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474368 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474372 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474376 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474381 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474386 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474390 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474394 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474398 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474402 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474406 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474410 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474414 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474418 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474421 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474427 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474434 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474439 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474443 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:54:28.093141 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474447 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474453 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474458 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474463 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474469 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474473 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474478 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474482 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474486 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474490 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474494 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474498 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474502 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474506 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474510 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474514 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474518 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:54:28.093753 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:27.474523 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.474531 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.474699 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.477920 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.479669 2558 server.go:1019] "Starting client certificate rotation" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.479770 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.479798 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.482602 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.483669 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.493039 2558 log.go:25] "Validated CRI v1 runtime API" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.499331 2558 log.go:25] "Validated CRI v1 image API" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.500147 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 14:54:28.094408 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.503266 2558 fs.go:135] Filesystem UUIDs: map[41fc5e4e-5b50-4195-9fe6-acdc9f4d746d:/dev/nvme0n1p3 4306d1f8-ce19-480f-9a5b-dfc16f3e02a1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 09 14:54:28.094843 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.503281 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.508802 2558 manager.go:216] Machine: {Timestamp:2026-07-09 14:54:27.507447564 +0000 UTC m=+0.370454723 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200175 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec258f3071186b5e81dd46acb6ef86c0 SystemUUID:ec258f30-7118-6b5e-81dd-46acb6ef86c0 BootID:065e7fb5-378b-4f6a-a3bb-46836edf01b4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:5e:f0:7a:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:5e:f0:7a:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:a7:e4:29:eb:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.508901 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.508973 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.512447 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.512474 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-231.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.513360 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.513369 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 14:54:28.095035 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.513376 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.513988 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.514212 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.515930 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519431 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519537 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519564 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519576 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519590 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519598 2558 kubelet.go:398] "Adding apiserver pod source" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519608 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519931 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.519946 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.522602 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525068 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525095 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525238 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525251 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525256 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525261 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525269 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525275 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525828 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525835 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525842 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525847 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525855 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.525863 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.526747 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.526753 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.529835 2558 server.go:1268] "Started kubelet" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.529922 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.529919 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.529989 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.531251 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.531365 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-231.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.531412 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.531417 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.531495 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 14:54:28.095424 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.536403 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.539313 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.539336 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.539400 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540000 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540021 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540000 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540122 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540129 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540202 2558 factory.go:55] Registering systemd factory Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540245 2558 factory.go:223] Registration of the systemd container factory successfully Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540447 2558 factory.go:153] Registering CRI-O factory Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540459 2558 factory.go:223] Registration of the crio container factory successfully Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540503 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540530 2558 factory.go:103] Registering Raw factory Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540543 2558 manager.go:1196] Started watching for new ooms in manager Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.540980 2558 manager.go:349] Starting recovery of all containers Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.539894 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a759b542 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.529798978 +0000 UTC m=+0.392806138,LastTimestamp:2026-07-09 14:54:27.529798978 +0000 UTC m=+0.392806138,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.546175 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.546270 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-231.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.549731 2558 manager.go:354] Recovery completed Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.553611 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555265 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555291 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555299 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555795 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555807 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555825 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.555808 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.556553 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.556578 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.556590 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.557085 2558 policy_none.go:49] "None policy: Start" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.557101 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 14:54:28.097091 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.557114 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.557595 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a8de8d04 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-231.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.55528218 +0000 UTC m=+0.418289339,LastTimestamp:2026-07-09 14:54:27.55528218 +0000 UTC m=+0.418289339,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.558204 2558 policy_none.go:47] "Start" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.569602 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a8debe07 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-231.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.555294727 +0000 UTC m=+0.418301886,LastTimestamp:2026-07-09 14:54:27.555294727 +0000 UTC m=+0.418301886,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.571554 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnczv" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.573317 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a8dedd37 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-231.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.555302711 +0000 UTC m=+0.418309874,LastTimestamp:2026-07-09 14:54:27.555302711 +0000 UTC m=+0.418309874,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.581135 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-231.ec2.internal.18c0a6d0a8de8d04\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a8de8d04 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-231.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.55528218 +0000 UTC m=+0.418289339,LastTimestamp:2026-07-09 14:54:27.556562413 +0000 UTC m=+0.419569572,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.098079 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.582516 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jnczv" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.588923 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-231.ec2.internal.18c0a6d0a8debe07\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-231.ec2.internal.18c0a6d0a8debe07 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-231.ec2.internal,UID:ip-10-0-142-231.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-231.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-231.ec2.internal,},FirstTimestamp:2026-07-09 14:54:27.555294727 +0000 UTC m=+0.418301886,LastTimestamp:2026-07-09 14:54:27.556584551 +0000 UTC m=+0.419591710,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-231.ec2.internal,}" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.615443 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.617287 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.617333 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.617364 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.617406 2558 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.620210 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.640380 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661429 2558 manager.go:337] "Starting Device Plugin manager" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.661463 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661476 2558 server.go:87] "Starting device plugin registration server" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661741 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661751 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661859 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661950 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.661959 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.662384 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.662425 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.717735 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal"] Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.717803 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.719497 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.719524 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.719539 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.720770 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.720908 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.720936 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721493 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721522 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721525 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.098321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721548 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721556 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.721533 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.722732 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.722760 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.723358 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.723383 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.723395 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.741242 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.741264 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.741278 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfc52fc8e6e2f32cc911f6a116c3e1f0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-231.ec2.internal\" (UID: \"dfc52fc8e6e2f32cc911f6a116c3e1f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.756822 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-231.ec2.internal\" not found" node="ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.758576 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-231.ec2.internal\" not found" node="ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.762758 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.762844 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-231.ec2.internal\" not found" node="ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.763580 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.763604 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.763613 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.763636 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.773163 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.773180 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-231.ec2.internal\": node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.786705 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.099137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfc52fc8e6e2f32cc911f6a116c3e1f0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-231.ec2.internal\" (UID: \"dfc52fc8e6e2f32cc911f6a116c3e1f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841562 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfc52fc8e6e2f32cc911f6a116c3e1f0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-231.ec2.internal\" (UID: \"dfc52fc8e6e2f32cc911f6a116c3e1f0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841467 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:27.841594 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fdcb10ea9147ea5f07b6a2b3da4efd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal\" (UID: \"79fdcb10ea9147ea5f07b6a2b3da4efd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.886995 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:27.987915 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.062472 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.067098 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.100013 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.094630 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.296234 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.195151 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.296234 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.295583 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.396109 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-231.ec2.internal\" not found" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.436629 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.439681 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.455933 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.458138 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.466334 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.479030 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.479151 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.479185 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.521409 2558 apiserver.go:52] "Watching apiserver" Jul 09 14:54:28.541502 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.539824 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.546803 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.549198 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jlq7p","kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg","openshift-dns/node-resolver-j4lnm","openshift-image-registry/node-ca-glqpm","openshift-multus/multus-additional-cni-plugins-rbc7t","openshift-multus/multus-pzvfm","openshift-network-operator/iptables-alerter-b98pk","kube-system/konnectivity-agent-d88j8","openshift-cluster-node-tuning-operator/tuned-bl6gx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal","openshift-multus/network-metrics-daemon-cz22s","openshift-network-diagnostics/network-check-target-qg8dx","openshift-ovn-kubernetes/ovnkube-node-sjdnn"] Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.550734 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.550792 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.551957 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.552895 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.553059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.554417 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.555667 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.555836 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.556216 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.556360 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bvxmp\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.556486 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.557401 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.557478 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.558295 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.558658 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.558665 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.560259 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.560660 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.560769 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.560944 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5sdv\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.561446 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.561957 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.562052 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dptzf\"" Jul 09 14:54:28.739158 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.562316 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-thncm\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.564297 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.565611 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.566781 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.567195 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.567210 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vbpgf\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.567985 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.568434 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.568586 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.568731 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qhkfp\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.568785 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.569231 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.569281 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.569699 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lzfpr\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.569976 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.570412 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.570630 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.570682 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.570778 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.570828 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-995vz\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.571603 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.571676 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.578004 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.578003 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.578089 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.578216 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f4bdd\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.579475 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.579499 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.579500 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 14:54:28.740167 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.581525 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-99mxf" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.585897 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 14:49:28 +0000 UTC" deadline="2027-12-27 15:14:34.70863209 +0000 UTC" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.585932 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12864h20m6.122703112s" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.592828 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-99mxf" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.593324 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.641403 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646055 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646078 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-netns\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646094 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-host\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-netns\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646126 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-netd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-script-lib\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646192 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9vg\" (UniqueName: \"kubernetes.io/projected/b15fa70c-505f-4d9d-8300-6c2991041fba-kube-api-access-vb9vg\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-etc-selinux\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646292 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/488409ee-00bf-48aa-bb9a-d16af490d712-agent-certs\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646323 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-conf\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741205 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646354 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4dk\" (UniqueName: \"kubernetes.io/projected/e274d73f-ef22-459d-9da5-16fb56c9d52f-kube-api-access-qx4dk\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-registration-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-device-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646498 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-sys-fs\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646544 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-kubelet\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-systemd\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-tuned\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646603 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646629 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e6cb45c-1646-42d3-b22f-89038504faf9-serviceca\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646657 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m55vs\" (UniqueName: \"kubernetes.io/projected/7e6cb45c-1646-42d3-b22f-89038504faf9-kube-api-access-m55vs\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-tmp\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646704 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntp76\" (UniqueName: \"kubernetes.io/projected/697d88da-9062-4432-aeb2-9cbb5a66299b-kube-api-access-ntp76\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-var-lib-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.741769 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646747 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e6cb45c-1646-42d3-b22f-89038504faf9-host\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646762 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-os-release\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646786 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cnibin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-os-release\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646829 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-socket-dir-parent\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646857 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-multus-certs\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646897 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mr4f\" (UniqueName: \"kubernetes.io/projected/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-kube-api-access-9mr4f\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.646994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-env-overrides\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-dbus\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647089 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-slash\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-run\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647148 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-log-socket\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647163 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-multus\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.742564 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a1d1e8-92a5-402a-be24-b296282972d7-host-slash\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-config\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647211 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-kubelet-config\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-system-cni-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647292 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cni-binary-copy\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647310 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-modprobe-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647341 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647372 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-ovn\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647404 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-bin\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-hostroot\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647436 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-systemd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647453 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-k8s-cni-cncf-io\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-bin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647491 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647507 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647529 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-socket-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647549 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mttgz\" (UniqueName: \"kubernetes.io/projected/daccd9a4-b16c-4c79-ab63-599825357461-kube-api-access-mttgz\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647573 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-kubernetes\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647592 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647607 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-kubelet\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647625 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-etc-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647639 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysconfig\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-sys\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647698 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647723 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-cnibin\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647753 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.743759 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647768 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-systemd-units\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647783 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a9f981-930d-4fbd-a6a7-43e8bf453208-hosts-file\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647805 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a9f981-930d-4fbd-a6a7-43e8bf453208-tmp-dir\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-daemon-config\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647839 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a1d1e8-92a5-402a-be24-b296282972d7-iptables-alerter-script\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqh8v\" (UniqueName: \"kubernetes.io/projected/d4a1d1e8-92a5-402a-be24-b296282972d7-kube-api-access-xqh8v\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647884 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-node-log\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647913 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/488409ee-00bf-48aa-bb9a-d16af490d712-konnectivity-ca\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647938 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf9nc\" (UniqueName: \"kubernetes.io/projected/f1a9f981-930d-4fbd-a6a7-43e8bf453208-kube-api-access-mf9nc\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-system-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.647986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-conf-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648009 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-etc-kubernetes\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648031 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-lib-modules\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648058 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-var-lib-kubelet\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.744324 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15fa70c-505f-4d9d-8300-6c2991041fba-ovn-node-metrics-cert\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.744928 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmh5q\" (UniqueName: \"kubernetes.io/projected/0c0019e7-682b-4773-b3ce-a4de72691a28-kube-api-access-jmh5q\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.744928 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648131 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.744928 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.648156 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748569 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-kubelet\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748600 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-etc-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysconfig\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-sys\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748692 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-kubelet\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748711 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748715 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysconfig\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748721 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748742 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-cnibin\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748774 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-cnibin\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748784 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-systemd-units\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.933813 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748776 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748799 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a9f981-930d-4fbd-a6a7-43e8bf453208-hosts-file\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-etc-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a9f981-930d-4fbd-a6a7-43e8bf453208-tmp-dir\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748745 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-sys\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748836 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-systemd-units\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748862 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f1a9f981-930d-4fbd-a6a7-43e8bf453208-hosts-file\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748870 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-daemon-config\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748917 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a1d1e8-92a5-402a-be24-b296282972d7-iptables-alerter-script\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqh8v\" (UniqueName: \"kubernetes.io/projected/d4a1d1e8-92a5-402a-be24-b296282972d7-kube-api-access-xqh8v\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.748962 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-node-log\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.748996 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/488409ee-00bf-48aa-bb9a-d16af490d712-konnectivity-ca\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.749065 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:54:29.249012333 +0000 UTC m=+2.112019507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:28.937440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf9nc\" (UniqueName: \"kubernetes.io/projected/f1a9f981-930d-4fbd-a6a7-43e8bf453208-kube-api-access-mf9nc\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749094 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-node-log\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749120 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-system-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-conf-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-etc-kubernetes\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749183 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1a9f981-930d-4fbd-a6a7-43e8bf453208-tmp-dir\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749196 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-lib-modules\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749220 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-var-lib-kubelet\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749246 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15fa70c-505f-4d9d-8300-6c2991041fba-ovn-node-metrics-cert\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749274 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmh5q\" (UniqueName: \"kubernetes.io/projected/0c0019e7-682b-4773-b3ce-a4de72691a28-kube-api-access-jmh5q\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749284 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-conf-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749288 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-system-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.938301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749377 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-netns\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-host\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-netns\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749455 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-netd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749478 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-script-lib\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9vg\" (UniqueName: \"kubernetes.io/projected/b15fa70c-505f-4d9d-8300-6c2991041fba-kube-api-access-vb9vg\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749529 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-etc-selinux\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749552 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/488409ee-00bf-48aa-bb9a-d16af490d712-agent-certs\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-conf\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4dk\" (UniqueName: \"kubernetes.io/projected/e274d73f-ef22-459d-9da5-16fb56c9d52f-kube-api-access-qx4dk\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749597 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a1d1e8-92a5-402a-be24-b296282972d7-iptables-alerter-script\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749611 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749620 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/488409ee-00bf-48aa-bb9a-d16af490d712-konnectivity-ca\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.939207 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-registration-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-device-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749686 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.749707 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749714 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-sys-fs\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749712 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-netns\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749247 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-etc-kubernetes\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749734 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-etc-selinux\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-host\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.749770 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:54:29.249753474 +0000 UTC m=+2.112760622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-kubelet\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749857 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-daemon-config\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749874 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-cni-dir\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749915 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-systemd\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-tuned\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.939772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749967 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.749994 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e6cb45c-1646-42d3-b22f-89038504faf9-serviceca\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750015 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-netd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750022 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-sysctl-conf\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750066 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-systemd\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m55vs\" (UniqueName: \"kubernetes.io/projected/7e6cb45c-1646-42d3-b22f-89038504faf9-kube-api-access-m55vs\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750072 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-kubelet\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750069 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-sys-fs\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750086 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-registration-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750097 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-tmp\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750129 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-device-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntp76\" (UniqueName: \"kubernetes.io/projected/697d88da-9062-4432-aeb2-9cbb5a66299b-kube-api-access-ntp76\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750184 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-script-lib\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750270 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-lib-modules\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.940606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750299 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-var-lib-kubelet\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750370 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-var-lib-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e6cb45c-1646-42d3-b22f-89038504faf9-host\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750428 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750432 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-var-lib-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750477 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-run-netns\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750485 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-os-release\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750515 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cnibin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750561 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e6cb45c-1646-42d3-b22f-89038504faf9-host\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750611 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-os-release\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e6cb45c-1646-42d3-b22f-89038504faf9-serviceca\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-os-release\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750649 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cnibin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-socket-dir-parent\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-multus-certs\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941321 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750734 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-os-release\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750767 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-multus-socket-dir-parent\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mr4f\" (UniqueName: \"kubernetes.io/projected/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-kube-api-access-9mr4f\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750835 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-env-overrides\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750856 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-multus-certs\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750862 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-dbus\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750934 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-dbus\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750947 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-slash\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.750981 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-run\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751005 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-log-socket\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751032 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-multus\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751057 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a1d1e8-92a5-402a-be24-b296282972d7-host-slash\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751078 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-run\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751082 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-config\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.941940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751139 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-kubelet-config\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-system-cni-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751188 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cni-binary-copy\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751211 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-modprobe-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751260 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-ovn\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751281 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-bin\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-hostroot\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-systemd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-k8s-cni-cncf-io\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-bin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751452 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-socket-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.942507 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751479 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mttgz\" (UniqueName: \"kubernetes.io/projected/daccd9a4-b16c-4c79-ab63-599825357461-kube-api-access-mttgz\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751508 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-kubernetes\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751654 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751711 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-hostroot\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-slash\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751791 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce2db69a-8723-42cc-8359-292bf2d8e76d-kubelet-config\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751824 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/daccd9a4-b16c-4c79-ab63-599825357461-system-cni-dir\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751853 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-ovnkube-config\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751917 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-log-socket\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.751978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-multus\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752186 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a1d1e8-92a5-402a-be24-b296282972d7-host-slash\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752194 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/daccd9a4-b16c-4c79-ab63-599825357461-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-openvswitch\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943236 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752259 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-cni-binary-copy\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752262 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752289 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-systemd\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752325 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-run-k8s-cni-cncf-io\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752354 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-modprobe-d\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752363 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-host-var-lib-cni-bin\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752340 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c0019e7-682b-4773-b3ce-a4de72691a28-socket-dir\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752405 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-run-ovn\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752449 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b15fa70c-505f-4d9d-8300-6c2991041fba-host-cni-bin\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.752499 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-kubernetes\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.753572 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b15fa70c-505f-4d9d-8300-6c2991041fba-env-overrides\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.753660 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-tmp\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.753744 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e274d73f-ef22-459d-9da5-16fb56c9d52f-etc-tuned\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.753769 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b15fa70c-505f-4d9d-8300-6c2991041fba-ovn-node-metrics-cert\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.757850 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/488409ee-00bf-48aa-bb9a-d16af490d712-agent-certs\") pod \"konnectivity-agent-d88j8\" (UID: \"488409ee-00bf-48aa-bb9a-d16af490d712\") " pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:28.943808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.767803 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9vg\" (UniqueName: \"kubernetes.io/projected/b15fa70c-505f-4d9d-8300-6c2991041fba-kube-api-access-vb9vg\") pod \"ovnkube-node-sjdnn\" (UID: \"b15fa70c-505f-4d9d-8300-6c2991041fba\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.771154 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.771180 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.771194 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:28.771267 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:54:29.271250747 +0000 UTC m=+2.134257915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.773129 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4dk\" (UniqueName: \"kubernetes.io/projected/e274d73f-ef22-459d-9da5-16fb56c9d52f-kube-api-access-qx4dk\") pod \"tuned-bl6gx\" (UID: \"e274d73f-ef22-459d-9da5-16fb56c9d52f\") " pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.773521 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mr4f\" (UniqueName: \"kubernetes.io/projected/5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff-kube-api-access-9mr4f\") pod \"multus-pzvfm\" (UID: \"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff\") " pod="openshift-multus/multus-pzvfm" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.773604 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmh5q\" (UniqueName: \"kubernetes.io/projected/0c0019e7-682b-4773-b3ce-a4de72691a28-kube-api-access-jmh5q\") pod \"aws-ebs-csi-driver-node-c45pg\" (UID: \"0c0019e7-682b-4773-b3ce-a4de72691a28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.774165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mttgz\" (UniqueName: \"kubernetes.io/projected/daccd9a4-b16c-4c79-ab63-599825357461-kube-api-access-mttgz\") pod \"multus-additional-cni-plugins-rbc7t\" (UID: \"daccd9a4-b16c-4c79-ab63-599825357461\") " pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.774580 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqh8v\" (UniqueName: \"kubernetes.io/projected/d4a1d1e8-92a5-402a-be24-b296282972d7-kube-api-access-xqh8v\") pod \"iptables-alerter-b98pk\" (UID: \"d4a1d1e8-92a5-402a-be24-b296282972d7\") " pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.774592 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf9nc\" (UniqueName: \"kubernetes.io/projected/f1a9f981-930d-4fbd-a6a7-43e8bf453208-kube-api-access-mf9nc\") pod \"node-resolver-j4lnm\" (UID: \"f1a9f981-930d-4fbd-a6a7-43e8bf453208\") " pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.775427 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m55vs\" (UniqueName: \"kubernetes.io/projected/7e6cb45c-1646-42d3-b22f-89038504faf9-kube-api-access-m55vs\") pod \"node-ca-glqpm\" (UID: \"7e6cb45c-1646-42d3-b22f-89038504faf9\") " pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.780896 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntp76\" (UniqueName: \"kubernetes.io/projected/697d88da-9062-4432-aeb2-9cbb5a66299b-kube-api-access-ntp76\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:28.944432 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.837856 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc52fc8e6e2f32cc911f6a116c3e1f0.slice/crio-9c2668f1e1c0c80b8817ccf459c2dfc2dfe38fa89c079cae51a50a8770189bf7 WatchSource:0}: Error finding container 9c2668f1e1c0c80b8817ccf459c2dfc2dfe38fa89c079cae51a50a8770189bf7: Status 404 returned error can't find the container with id 9c2668f1e1c0c80b8817ccf459c2dfc2dfe38fa89c079cae51a50a8770189bf7 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.838180 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fdcb10ea9147ea5f07b6a2b3da4efd.slice/crio-a15c3b1601a721a03bb27f2f886a7b6b1b46f9d09513c0749be4eabaecfc465a WatchSource:0}: Error finding container a15c3b1601a721a03bb27f2f886a7b6b1b46f9d09513c0749be4eabaecfc465a: Status 404 returned error can't find the container with id a15c3b1601a721a03bb27f2f886a7b6b1b46f9d09513c0749be4eabaecfc465a Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.843704 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.885493 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.891994 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4lnm" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.894890 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0019e7_682b_4773_b3ce_a4de72691a28.slice/crio-b56d069646132ae21f771857ba7a1b055664ec3fedf229d37f427a705f061fa5 WatchSource:0}: Error finding container b56d069646132ae21f771857ba7a1b055664ec3fedf229d37f427a705f061fa5: Status 404 returned error can't find the container with id b56d069646132ae21f771857ba7a1b055664ec3fedf229d37f427a705f061fa5 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.899849 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a9f981_930d_4fbd_a6a7_43e8bf453208.slice/crio-4b86cff360f8e9f4a3435beda08b297e568af4a6173ee5a42963645d0472ba7e WatchSource:0}: Error finding container 4b86cff360f8e9f4a3435beda08b297e568af4a6173ee5a42963645d0472ba7e: Status 404 returned error can't find the container with id 4b86cff360f8e9f4a3435beda08b297e568af4a6173ee5a42963645d0472ba7e Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.909731 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.910767 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-glqpm" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.925116 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6cb45c_1646_42d3_b22f_89038504faf9.slice/crio-17676d89da7f3da0b39a91f78215050bce0ca4ace27d003acb1f12b3b25b8025 WatchSource:0}: Error finding container 17676d89da7f3da0b39a91f78215050bce0ca4ace27d003acb1f12b3b25b8025: Status 404 returned error can't find the container with id 17676d89da7f3da0b39a91f78215050bce0ca4ace27d003acb1f12b3b25b8025 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.930097 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.938922 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pzvfm" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.941723 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaccd9a4_b16c_4c79_ab63_599825357461.slice/crio-e8f9ddcb35da0cd4d20f1e566c5da7ef18dcdfd0810c59866daa1e0a9a2163e9 WatchSource:0}: Error finding container e8f9ddcb35da0cd4d20f1e566c5da7ef18dcdfd0810c59866daa1e0a9a2163e9: Status 404 returned error can't find the container with id e8f9ddcb35da0cd4d20f1e566c5da7ef18dcdfd0810c59866daa1e0a9a2163e9 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.947395 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbaa3b3_a708_4a5f_95f3_cecf09bb3fff.slice/crio-f0032ed034a35ea43a7db1d6d5ab02809be594054b70910ee1b8f4972c0e6e08 WatchSource:0}: Error finding container f0032ed034a35ea43a7db1d6d5ab02809be594054b70910ee1b8f4972c0e6e08: Status 404 returned error can't find the container with id f0032ed034a35ea43a7db1d6d5ab02809be594054b70910ee1b8f4972c0e6e08 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.952290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b98pk" Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.961254 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a1d1e8_92a5_402a_be24_b296282972d7.slice/crio-d76502041314bd6360588a03f5a4b7f9c5ac2c8c52fe3f1e2d225a87e5c611a2 WatchSource:0}: Error finding container d76502041314bd6360588a03f5a4b7f9c5ac2c8c52fe3f1e2d225a87e5c611a2: Status 404 returned error can't find the container with id d76502041314bd6360588a03f5a4b7f9c5ac2c8c52fe3f1e2d225a87e5c611a2 Jul 09 14:54:29.209008 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.967305 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:28.975352 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488409ee_00bf_48aa_bb9a_d16af490d712.slice/crio-14240889db9640662f8e5f7c09138cee5b481e27f85bd476088016554027b5a8 WatchSource:0}: Error finding container 14240889db9640662f8e5f7c09138cee5b481e27f85bd476088016554027b5a8: Status 404 returned error can't find the container with id 14240889db9640662f8e5f7c09138cee5b481e27f85bd476088016554027b5a8 Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.990474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:28.995392 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:54:29.007537 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15fa70c_505f_4d9d_8300_6c2991041fba.slice/crio-773d35136a1ee50a8d5b91c17a7d00693d839e9b7b469f8276ad9313215a2aa7 WatchSource:0}: Error finding container 773d35136a1ee50a8d5b91c17a7d00693d839e9b7b469f8276ad9313215a2aa7: Status 404 returned error can't find the container with id 773d35136a1ee50a8d5b91c17a7d00693d839e9b7b469f8276ad9313215a2aa7 Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.009407 2558 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-sjdnn_openshift-ovn-kubernetes(b15fa70c-505f-4d9d-8300-6c2991041fba): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.010579 2558 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-sjdnn_openshift-ovn-kubernetes(b15fa70c-505f-4d9d-8300-6c2991041fba): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 09 14:54:29.211500 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.013389 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d1573db16143027c8ffaf46a01ce12f2bcd89c15410c42ca54be4b3b1ddfdd6\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" podUID="b15fa70c-505f-4d9d-8300-6c2991041fba" Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.255457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.255504 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.255627 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.255684 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:54:30.255671257 +0000 UTC m=+3.118678405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.255627 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.255761 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:54:30.255749181 +0000 UTC m=+3.118756339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.328782 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.356801 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.356962 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.356977 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.356986 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:29.465297 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.357055 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:54:30.357037427 +0000 UTC m=+3.220044586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.594391 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:49:29 +0000 UTC" deadline="2028-04-21 11:13:38.866163086 +0000 UTC" Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.594453 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15644h19m9.271715319s" Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.618161 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.618316 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.626953 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"773d35136a1ee50a8d5b91c17a7d00693d839e9b7b469f8276ad9313215a2aa7"} Jul 09 14:54:29.635940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.631155 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d88j8" event={"ID":"488409ee-00bf-48aa-bb9a-d16af490d712","Type":"ContainerStarted","Data":"14240889db9640662f8e5f7c09138cee5b481e27f85bd476088016554027b5a8"} Jul 09 14:54:29.638497 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.637819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pzvfm" event={"ID":"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff","Type":"ContainerStarted","Data":"f0032ed034a35ea43a7db1d6d5ab02809be594054b70910ee1b8f4972c0e6e08"} Jul 09 14:54:29.640179 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.639702 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerStarted","Data":"e8f9ddcb35da0cd4d20f1e566c5da7ef18dcdfd0810c59866daa1e0a9a2163e9"} Jul 09 14:54:29.646027 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.642633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-glqpm" event={"ID":"7e6cb45c-1646-42d3-b22f-89038504faf9","Type":"ContainerStarted","Data":"17676d89da7f3da0b39a91f78215050bce0ca4ace27d003acb1f12b3b25b8025"} Jul 09 14:54:29.652823 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.648432 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" event={"ID":"0c0019e7-682b-4773-b3ce-a4de72691a28","Type":"ContainerStarted","Data":"b56d069646132ae21f771857ba7a1b055664ec3fedf229d37f427a705f061fa5"} Jul 09 14:54:29.660497 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.659075 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" event={"ID":"79fdcb10ea9147ea5f07b6a2b3da4efd","Type":"ContainerStarted","Data":"a15c3b1601a721a03bb27f2f886a7b6b1b46f9d09513c0749be4eabaecfc465a"} Jul 09 14:54:29.765026 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.706273 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" event={"ID":"e274d73f-ef22-459d-9da5-16fb56c9d52f","Type":"ContainerStarted","Data":"3c3c1952857f170e540d69d1dd794ab2c48669e8d9a8255c1237510b126527ba"} Jul 09 14:54:29.765026 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.713397 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b98pk" event={"ID":"d4a1d1e8-92a5-402a-be24-b296282972d7","Type":"ContainerStarted","Data":"d76502041314bd6360588a03f5a4b7f9c5ac2c8c52fe3f1e2d225a87e5c611a2"} Jul 09 14:54:29.765026 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.715358 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4lnm" event={"ID":"f1a9f981-930d-4fbd-a6a7-43e8bf453208","Type":"ContainerStarted","Data":"4b86cff360f8e9f4a3435beda08b297e568af4a6173ee5a42963645d0472ba7e"} Jul 09 14:54:29.765026 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:29.718442 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" event={"ID":"dfc52fc8e6e2f32cc911f6a116c3e1f0","Type":"ContainerStarted","Data":"9c2668f1e1c0c80b8817ccf459c2dfc2dfe38fa89c079cae51a50a8770189bf7"} Jul 09 14:54:29.765026 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:29.729758 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d1573db16143027c8ffaf46a01ce12f2bcd89c15410c42ca54be4b3b1ddfdd6\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9d1573db16143027c8ffaf46a01ce12f2bcd89c15410c42ca54be4b3b1ddfdd6\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc4c36822d1ce0f4acab78208d171b757bca1285f69c9653d5a959fa853d8470\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" podUID="b15fa70c-505f-4d9d-8300-6c2991041fba" Jul 09 14:54:30.319061 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.319028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:30.319201 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.319098 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:30.319307 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.319276 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:30.319369 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.319343 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:54:32.319324852 +0000 UTC m=+5.182332015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:30.319434 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.319408 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:30.319489 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.319442 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:54:32.319430079 +0000 UTC m=+5.182437230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:30.420655 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.420588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:30.420829 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.420778 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:30.420829 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.420799 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:30.420829 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.420813 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:30.421019 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.420870 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:54:32.420852179 +0000 UTC m=+5.283859331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:30.595245 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.595099 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:49:29 +0000 UTC" deadline="2028-02-04 04:11:08.214345753 +0000 UTC" Jul 09 14:54:30.595245 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.595161 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13789h16m37.619188748s" Jul 09 14:54:30.618544 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.618447 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:30.618544 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:30.618492 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:30.618776 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.618586 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:30.619081 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:30.618983 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:31.269653 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:31.269624 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:54:31.622313 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:31.622239 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:31.622469 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:31.622367 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:32.339645 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:32.339550 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:32.339645 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:32.339621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:32.340409 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.339753 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:32.340409 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.339832 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:54:36.339810594 +0000 UTC m=+9.202817755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:32.340409 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.339763 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:32.340409 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.339911 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:54:36.339893197 +0000 UTC m=+9.202900351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:32.440416 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:32.440185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:32.440416 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.440392 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:32.440416 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.440414 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:32.440416 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.440426 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:32.440754 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.440490 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:54:36.44047045 +0000 UTC m=+9.303477600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:32.618042 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:32.617960 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:32.618195 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:32.617960 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:32.618195 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.618081 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:32.618357 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:32.618206 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:33.617665 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:33.617630 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:33.618303 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:33.617776 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:34.617567 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:34.617530 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:34.617722 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:34.617530 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:34.618064 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:34.617837 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:34.618141 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:34.618101 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:35.618351 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:35.618321 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:35.618769 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:35.618458 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:36.372854 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:36.372765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:36.372854 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:36.372819 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:36.373085 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.372952 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:36.373085 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.372958 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:36.373085 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.373014 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:54:44.372998224 +0000 UTC m=+17.236005384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:36.373085 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.373033 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:54:44.373023367 +0000 UTC m=+17.236030516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:36.474201 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:36.474170 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:36.474341 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.474318 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:36.474403 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.474347 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:36.474403 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.474360 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:36.474507 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.474423 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:54:44.474404275 +0000 UTC m=+17.337411423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:36.618219 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:36.618188 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:36.618395 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:36.618196 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:36.618395 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.618293 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:36.618395 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:36.618384 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:37.621556 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:37.621529 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:37.621966 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:37.621653 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:38.617900 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.617659 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:38.618050 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:38.618008 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:38.618128 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.617709 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:38.618215 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:38.618194 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:38.747742 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.747704 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d88j8" event={"ID":"488409ee-00bf-48aa-bb9a-d16af490d712","Type":"ContainerStarted","Data":"3b8a3de7533463f133a5d0b9da4b62ff407b967c675149f4837ea22c27bb75af"} Jul 09 14:54:38.749405 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.749380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerStarted","Data":"4eddbcdef59e796e1f9f648936fe31d9c7f78e0207446ce7f8b4333405f75912"} Jul 09 14:54:38.751003 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.750974 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-glqpm" event={"ID":"7e6cb45c-1646-42d3-b22f-89038504faf9","Type":"ContainerStarted","Data":"1e6fe59128565cf8e3b238e7ccb5e2235ade971ea2e62c9be97bc95e84aa8c3e"} Jul 09 14:54:38.752658 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.752618 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" event={"ID":"0c0019e7-682b-4773-b3ce-a4de72691a28","Type":"ContainerStarted","Data":"1d43dcfc02264967e5cbe11dc24bb0d5bca7a797705a953f56945cfb112b56a1"} Jul 09 14:54:38.753963 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.753943 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" event={"ID":"79fdcb10ea9147ea5f07b6a2b3da4efd","Type":"ContainerStarted","Data":"adbc18148b0213a77ae569b86f6b70e5e580881e4c93effb33495bde5a959c87"} Jul 09 14:54:38.755514 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.755493 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" event={"ID":"e274d73f-ef22-459d-9da5-16fb56c9d52f","Type":"ContainerStarted","Data":"966c01df50ad5c9fd61d1631238b8e38542f2dddfc45199d08fdcb74774356d6"} Jul 09 14:54:38.756955 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.756937 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4lnm" event={"ID":"f1a9f981-930d-4fbd-a6a7-43e8bf453208","Type":"ContainerStarted","Data":"a0c386775e152eac16a4be597072f085bfb7fdd5cf5056b9deace447c227db3e"} Jul 09 14:54:38.758166 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.758147 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" event={"ID":"dfc52fc8e6e2f32cc911f6a116c3e1f0","Type":"ContainerStarted","Data":"cd76cf22b434d6f529c13bcc8b49804fbf808ec8ed4d9963b3887d838643f5c8"} Jul 09 14:54:38.767259 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.767221 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d88j8" podStartSLOduration=1.308814556 podStartE2EDuration="10.767199533s" totalImagesPullingTime="9.458384977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:38.766846239 +0000 UTC m=+11.629853408" watchObservedRunningTime="2026-07-09 14:54:38.767199533 +0000 UTC m=+11.630206701" Jul 09 14:54:38.781123 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.781083 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j4lnm" podStartSLOduration=1.247242212 podStartE2EDuration="10.781070971s" totalImagesPullingTime="9.533828759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:38.780868322 +0000 UTC m=+11.643875522" watchObservedRunningTime="2026-07-09 14:54:38.781070971 +0000 UTC m=+11.644078139" Jul 09 14:54:38.837354 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.837304 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-glqpm" podStartSLOduration=1.3551822 podStartE2EDuration="10.83728385s" totalImagesPullingTime="9.48210165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:38.836685493 +0000 UTC m=+11.699692662" watchObservedRunningTime="2026-07-09 14:54:38.83728385 +0000 UTC m=+11.700291025" Jul 09 14:54:38.854017 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:38.853976 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bl6gx" podStartSLOduration=1.400804992 podStartE2EDuration="10.853962188s" totalImagesPullingTime="9.453157196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:38.853633675 +0000 UTC m=+11.716640844" watchObservedRunningTime="2026-07-09 14:54:38.853962188 +0000 UTC m=+11.716969356" Jul 09 14:54:39.621795 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.621597 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:39.621980 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:39.621925 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:39.762309 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.762279 2558 generic.go:356] "Generic (PLEG): container finished" podID="79fdcb10ea9147ea5f07b6a2b3da4efd" containerID="adbc18148b0213a77ae569b86f6b70e5e580881e4c93effb33495bde5a959c87" exitCode=0 Jul 09 14:54:39.762731 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.762372 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" event={"ID":"79fdcb10ea9147ea5f07b6a2b3da4efd","Type":"ContainerDied","Data":"adbc18148b0213a77ae569b86f6b70e5e580881e4c93effb33495bde5a959c87"} Jul 09 14:54:39.764141 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.764088 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="4eddbcdef59e796e1f9f648936fe31d9c7f78e0207446ce7f8b4333405f75912" exitCode=0 Jul 09 14:54:39.764141 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.764120 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"4eddbcdef59e796e1f9f648936fe31d9c7f78e0207446ce7f8b4333405f75912"} Jul 09 14:54:39.778496 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.778451 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-231.ec2.internal" podStartSLOduration=11.778432364 podStartE2EDuration="11.778432364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:38.868362989 +0000 UTC m=+11.731370187" watchObservedRunningTime="2026-07-09 14:54:39.778432364 +0000 UTC m=+12.641439537" Jul 09 14:54:39.839457 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:39.839431 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 14:54:40.506206 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.506172 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:40.618231 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.618195 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:40.618389 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:40.618324 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:40.618389 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.618382 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:40.618522 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:40.618502 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:40.709048 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.708928 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T14:54:39.839451512Z","UUID":"42f6cf83-fa4e-4ee5-b46d-f875e1e1d3bd","Handler":null,"Name":"","Endpoint":""} Jul 09 14:54:40.710494 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.710468 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 14:54:40.710620 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.710509 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 14:54:40.768308 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.768211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b98pk" event={"ID":"d4a1d1e8-92a5-402a-be24-b296282972d7","Type":"ContainerStarted","Data":"cb9a6efe4fcf0c626602104e1eb33e46d6f41d5a320cd983163481aec3dc00ad"} Jul 09 14:54:40.770244 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.770216 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" event={"ID":"0c0019e7-682b-4773-b3ce-a4de72691a28","Type":"ContainerStarted","Data":"5af1e998767f9fdeef9625fab95f5194b5431d61627fe0211a9b2d6ca37ec65f"} Jul 09 14:54:40.772071 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.772045 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" event={"ID":"79fdcb10ea9147ea5f07b6a2b3da4efd","Type":"ContainerStarted","Data":"7e33da64164246dc0c0a25ab38b6b7077cd17966ff93a136a0fa675f9e0910f4"} Jul 09 14:54:40.785864 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.785814 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b98pk" podStartSLOduration=3.311602724 podStartE2EDuration="12.78579816s" totalImagesPullingTime="9.474195436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:40.785505099 +0000 UTC m=+13.648512265" watchObservedRunningTime="2026-07-09 14:54:40.78579816 +0000 UTC m=+13.648805327" Jul 09 14:54:40.801128 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.801081 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-231.ec2.internal" podStartSLOduration=12.801062824 podStartE2EDuration="12.801062824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:40.800589146 +0000 UTC m=+13.663596332" watchObservedRunningTime="2026-07-09 14:54:40.801062824 +0000 UTC m=+13.664069995" Jul 09 14:54:40.890638 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.890603 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:40.891307 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:40.891283 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:41.620722 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:41.620692 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:41.620938 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:41.620812 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:41.774742 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:41.774716 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d88j8" Jul 09 14:54:42.617792 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:42.617758 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:42.617977 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:42.617758 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:42.617977 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:42.617911 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:42.617977 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:42.617949 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:43.621169 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:43.621138 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:43.621811 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:43.621281 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:44.437541 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.437315 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:44.437803 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.437573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:44.437803 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.437703 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:44.437803 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.437759 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:00.437740118 +0000 UTC m=+33.300747284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:54:44.438172 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.438154 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:44.438234 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.438206 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:00.4381918 +0000 UTC m=+33.301198950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:54:44.538266 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.538234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:44.538394 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.538379 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:54:44.538467 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.538399 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:54:44.538467 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.538412 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:44.538467 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.538462 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:00.538444353 +0000 UTC m=+33.401451514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:54:44.618126 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.618094 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:44.618289 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.618227 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:44.618382 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.618359 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:44.618492 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:44.618470 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:44.782018 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.781980 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pzvfm" event={"ID":"5fbaa3b3-a708-4a5f-95f3-cecf09bb3fff","Type":"ContainerStarted","Data":"491f3cb273e9349dfc101aa58f1133ceae048ed408f08e4c809624ff5f3d41f7"} Jul 09 14:54:44.784426 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.784370 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" event={"ID":"0c0019e7-682b-4773-b3ce-a4de72691a28","Type":"ContainerStarted","Data":"efeb73a9435bc4e87deda020ec1417a8914a3b7002f5b2b4f2ad469f8e562f67"} Jul 09 14:54:44.800243 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.800196 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pzvfm" podStartSLOduration=1.8740770869999999 podStartE2EDuration="16.800177911s" totalImagesPullingTime="14.926100824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:44.798680285 +0000 UTC m=+17.661687476" watchObservedRunningTime="2026-07-09 14:54:44.800177911 +0000 UTC m=+17.663185079" Jul 09 14:54:44.814653 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:44.814617 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c45pg" podStartSLOduration=1.704884273 podStartE2EDuration="16.814605111s" totalImagesPullingTime="15.109720838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:54:44.814449659 +0000 UTC m=+17.677456827" watchObservedRunningTime="2026-07-09 14:54:44.814605111 +0000 UTC m=+17.677612281" Jul 09 14:54:45.618074 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:45.618043 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:45.618231 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:45.618170 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:46.618618 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:46.618585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:46.618618 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:46.618600 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:46.619135 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:46.618731 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:46.619135 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:46.618971 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:47.618955 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:47.618922 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:47.619390 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:47.619009 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:48.617629 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:48.617596 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:48.617803 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:48.617726 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:48.617803 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:48.617754 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:48.618091 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:48.617902 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:49.617926 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:49.617887 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:49.618292 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:49.617996 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:49.797936 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:49.797904 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="8c584ac58b2adb0e256324883ede1b874e6033ca497da2809b2155b2762a4c77" exitCode=0 Jul 09 14:54:49.798093 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:49.797945 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"8c584ac58b2adb0e256324883ede1b874e6033ca497da2809b2155b2762a4c77"} Jul 09 14:54:50.618443 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:50.618406 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:50.618921 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:50.618537 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:50.618921 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:50.618589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:50.618921 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:50.618704 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:51.618496 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:51.618462 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:51.618926 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:51.618595 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:51.807138 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:51.807097 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="43d3700ddb88908ee16b410c784ad4a67208172fb5eeebca989f3df7720e8695" exitCode=0 Jul 09 14:54:51.807314 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:51.807146 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"43d3700ddb88908ee16b410c784ad4a67208172fb5eeebca989f3df7720e8695"} Jul 09 14:54:52.618000 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:52.617963 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:52.618159 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:52.617977 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:52.618159 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:52.618091 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:52.618253 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:52.618163 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:53.617747 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:53.617717 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:53.618224 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:53.617847 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:54.617761 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:54.617727 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:54.618188 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:54.617767 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:54.618188 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:54.617835 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:54.618188 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:54.617988 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:55.617795 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:55.617758 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:55.618366 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:55.617911 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:56.618582 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:56.618549 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:56.619029 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:56.618652 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:56.619029 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:56.618707 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:56.619029 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:56.618835 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:57.621636 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.619297 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:57.621636 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:57.619722 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:57.822915 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.822827 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:54:57.823145 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823123 2558 generic.go:356] "Generic (PLEG): container finished" podID="b15fa70c-505f-4d9d-8300-6c2991041fba" containerID="9cfc95d9d67ebcb5176bc2edebadbdf98909f65d2b05379334616b93efe91330" exitCode=1 Jul 09 14:54:57.823217 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"141f83388999cb2b32c6f960fcc1b28e0c3a7cf91ef08ce0e9b6ec10906b8659"} Jul 09 14:54:57.823280 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823265 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"973106ad38e48d70ee4fcecff5ace2ac8cc997dba66a3c0ab1bb805c1a0237e3"} Jul 09 14:54:57.823318 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823304 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"88c670e6dd5616bf83197878e3ca3406492988c8a4c4c6361d16d945e8453318"} Jul 09 14:54:57.823351 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823333 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"093b5ec58006ae0ad440cc242a9526fe01169627c52ceb8e58d6fd8fb7d880cf"} Jul 09 14:54:57.823398 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823370 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerDied","Data":"9cfc95d9d67ebcb5176bc2edebadbdf98909f65d2b05379334616b93efe91330"} Jul 09 14:54:57.823434 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.823399 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"6e408e764e9cd868f41e64221ea735d8c71e37cb8b17448b4e05631453e0c6da"} Jul 09 14:54:57.825395 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.825374 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="211eda9767a751d251d761864c55957e0edf905cd2cc1130ccca0c26d94659ad" exitCode=0 Jul 09 14:54:57.825436 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:57.825407 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"211eda9767a751d251d761864c55957e0edf905cd2cc1130ccca0c26d94659ad"} Jul 09 14:54:58.618481 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:58.618444 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:54:58.618644 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:58.618444 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:54:58.618644 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:58.618572 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:54:58.618644 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:58.618636 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:54:59.618627 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:59.618585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:54:59.619253 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:54:59.618697 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:54:59.834210 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:59.834176 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:54:59.834708 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:54:59.834683 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"501c8166cf6d1d4ab0fe9e3c3fd553f02e3d3d6fa383ee255d5b9a0fbc170f13"} Jul 09 14:55:00.454060 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:00.454024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:00.454223 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:00.454070 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:00.454223 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.454160 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:55:00.454223 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.454182 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:55:00.454223 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.454220 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret podName:ce2db69a-8723-42cc-8359-292bf2d8e76d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:32.454206279 +0000 UTC m=+65.317213425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret") pod "global-pull-secret-syncer-jlq7p" (UID: "ce2db69a-8723-42cc-8359-292bf2d8e76d") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:55:00.454356 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.454246 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:32.454232997 +0000 UTC m=+65.317240146 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:55:00.554457 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:00.554425 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:00.554609 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.554543 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:55:00.554609 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.554558 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:55:00.554609 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.554566 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rqxg9 for pod openshift-network-diagnostics/network-check-target-qg8dx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:55:00.554703 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.554619 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9 podName:c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:32.554605589 +0000 UTC m=+65.417612739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqxg9" (UniqueName: "kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9") pod "network-check-target-qg8dx" (UID: "c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:55:00.618317 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:00.618286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:00.618317 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:00.618322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:00.618480 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.618398 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:00.618528 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:00.618506 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:01.620763 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:01.620735 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:01.621139 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:01.620825 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:55:02.618575 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:02.618540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:02.618575 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:02.618543 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:02.618797 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:02.618675 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:02.618847 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:02.618814 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:03.618551 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.618325 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:03.618951 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:03.618558 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:55:03.847504 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.847481 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:55:03.847808 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.847786 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"d1d5e5ba6af795f74381d77443b06535be18510a915ed38ce3abe2bb3fe5e30f"} Jul 09 14:55:03.848043 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.848020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:03.848249 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.848223 2558 scope.go:117] "RemoveContainer" containerID="9cfc95d9d67ebcb5176bc2edebadbdf98909f65d2b05379334616b93efe91330" Jul 09 14:55:03.850656 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.850639 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="ccabcdea286e15c3ae60cfa16f70152421eacb1444b4d2673adc3ef345729b8a" exitCode=0 Jul 09 14:55:03.850736 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.850672 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"ccabcdea286e15c3ae60cfa16f70152421eacb1444b4d2673adc3ef345729b8a"} Jul 09 14:55:03.865763 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:03.865743 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:04.617932 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.617845 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:04.618061 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:04.617959 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:04.618061 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.618012 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:04.618121 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:04.618105 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:04.856590 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.856566 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:55:04.857039 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.856905 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" event={"ID":"b15fa70c-505f-4d9d-8300-6c2991041fba","Type":"ContainerStarted","Data":"314cf71c3c5bdb3131eb84ce184742da6c432633d32cbe98921825ad5273d55f"} Jul 09 14:55:04.857140 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.857124 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:04.857214 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.857202 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:04.859840 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.859815 2558 generic.go:356] "Generic (PLEG): container finished" podID="daccd9a4-b16c-4c79-ab63-599825357461" containerID="3a29af9cfede41c0d44ce869d8bba83b7e44af313463919794147f56ab7205de" exitCode=0 Jul 09 14:55:04.859965 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.859845 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerDied","Data":"3a29af9cfede41c0d44ce869d8bba83b7e44af313463919794147f56ab7205de"} Jul 09 14:55:04.871917 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.871855 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:04.915087 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:04.915049 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" podStartSLOduration=8.935487247 podStartE2EDuration="36.915032309s" totalImagesPullingTime="27.979545062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-09 14:55:04.890098652 +0000 UTC m=+37.753105820" watchObservedRunningTime="2026-07-09 14:55:04.915032309 +0000 UTC m=+37.778039478" Jul 09 14:55:05.267519 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.267491 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jlq7p"] Jul 09 14:55:05.267648 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.267612 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:05.267700 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:05.267684 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:55:05.270305 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.270284 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qg8dx"] Jul 09 14:55:05.270410 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.270361 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:05.270448 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:05.270424 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:05.273407 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.273383 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cz22s"] Jul 09 14:55:05.273498 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.273484 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:05.273595 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:05.273578 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:05.865056 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.865020 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" event={"ID":"daccd9a4-b16c-4c79-ab63-599825357461","Type":"ContainerStarted","Data":"db8675840e703eb1901dbaa3ba42682f4e86de92a250f8fe13d7408aa13507f9"} Jul 09 14:55:05.894360 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:05.894318 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rbc7t" podStartSLOduration=7.860783419 podStartE2EDuration="37.894302581s" totalImagesPullingTime="30.033519162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:05.894142296 +0000 UTC m=+38.757149463" watchObservedRunningTime="2026-07-09 14:55:05.894302581 +0000 UTC m=+38.757309749" Jul 09 14:55:06.618403 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:06.618186 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:06.618525 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:06.618209 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:06.618525 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:06.618430 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:55:06.618525 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:06.618214 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:06.618525 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:06.618501 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:06.618652 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:06.618568 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:08.617545 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:08.617520 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:08.617925 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:08.617520 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:08.617925 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:08.617520 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:08.617925 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:08.617711 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:55:08.617925 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:08.617608 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jlq7p" podUID="ce2db69a-8723-42cc-8359-292bf2d8e76d" Jul 09 14:55:08.617925 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:08.617778 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg8dx" podUID="c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183" Jul 09 14:55:09.419921 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.419894 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-231.ec2.internal" event="NodeReady" Jul 09 14:55:09.420069 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.419992 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 14:55:09.461172 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.461142 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hqmnq"] Jul 09 14:55:09.482826 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.482785 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z"] Jul 09 14:55:09.482989 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.482943 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.486323 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.486160 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 14:55:09.486323 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.486173 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 14:55:09.486323 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.486305 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z59h7\"" Jul 09 14:55:09.501498 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.501473 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:55:09.501615 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.501580 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.504814 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.504764 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 14:55:09.504814 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.504791 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 14:55:09.505001 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.504819 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w5kxs\"" Jul 09 14:55:09.505001 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.504792 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 14:55:09.505133 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.505119 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 14:55:09.520919 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.520896 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z"] Jul 09 14:55:09.521006 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.520942 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.524273 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.524256 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 14:55:09.524622 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.524607 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 14:55:09.525378 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.525360 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vdfrd\"" Jul 09 14:55:09.525470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.525387 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 14:55:09.532331 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.532314 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 14:55:09.548217 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.548193 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9"] Jul 09 14:55:09.548312 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.548297 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.551575 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.551560 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 14:55:09.568306 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568283 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hqmnq"] Jul 09 14:55:09.568399 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568323 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z"] Jul 09 14:55:09.568399 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568342 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z"] Jul 09 14:55:09.568399 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568361 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9"] Jul 09 14:55:09.568399 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568381 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zz5xw"] Jul 09 14:55:09.568399 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.568396 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.572506 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.572487 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 14:55:09.572772 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.572754 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 14:55:09.572966 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.572951 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 14:55:09.573168 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.573154 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 14:55:09.586355 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.586339 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dgnf8"] Jul 09 14:55:09.586483 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.586469 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.589536 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.589515 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 14:55:09.589606 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.589581 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 14:55:09.589989 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.589977 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rns85\"" Jul 09 14:55:09.607182 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.607155 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:55:09.607247 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.607193 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dgnf8"] Jul 09 14:55:09.607247 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.607213 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zz5xw"] Jul 09 14:55:09.607310 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.607268 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.611615 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.611601 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 14:55:09.611930 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.611912 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pfsrh\"" Jul 09 14:55:09.611930 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.611927 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 14:55:09.612057 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.612036 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 14:55:09.618938 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.618920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.618945 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a8677844-41dc-4c2c-b82a-6ecf37554f9d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.618964 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.618980 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.618999 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619193 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4vr8\" (UniqueName: \"kubernetes.io/projected/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-kube-api-access-p4vr8\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7129d9f0-9ebf-4900-8622-8ea78d986a22-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619329 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619424 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a665b593-b6bf-4a32-a698-217aea9b6844-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619429 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7129d9f0-9ebf-4900-8622-8ea78d986a22-tmp\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619464 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619498 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619528 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9jz\" (UniqueName: \"kubernetes.io/projected/7129d9f0-9ebf-4900-8622-8ea78d986a22-kube-api-access-4t9jz\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619575 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wljgp\" (UniqueName: \"kubernetes.io/projected/a665b593-b6bf-4a32-a698-217aea9b6844-kube-api-access-wljgp\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619612 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.619850 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.619673 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbvs4\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.691729 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.691690 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:09.720628 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720603 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2010273d-2add-4376-9b00-452a615ad1ce-cert\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720654 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720697 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720735 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbvs4\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720760 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a8677844-41dc-4c2c-b82a-6ecf37554f9d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720807 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.720869 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720848 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7129d9f0-9ebf-4900-8622-8ea78d986a22-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.720903 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a665b593-b6bf-4a32-a698-217aea9b6844-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721079 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721129 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1078985e-6c76-4495-b4f0-43cd7eeb645b-tmp-dir\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721171 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721198 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7129d9f0-9ebf-4900-8622-8ea78d986a22-tmp\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721219 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmcg\" (UniqueName: \"kubernetes.io/projected/2010273d-2add-4376-9b00-452a615ad1ce-kube-api-access-rrmcg\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721248 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9jz\" (UniqueName: \"kubernetes.io/projected/7129d9f0-9ebf-4900-8622-8ea78d986a22-kube-api-access-4t9jz\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.721271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721272 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wljgp\" (UniqueName: \"kubernetes.io/projected/a665b593-b6bf-4a32-a698-217aea9b6844-kube-api-access-wljgp\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721324 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721404 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721408 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4vr8\" (UniqueName: \"kubernetes.io/projected/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-kube-api-access-p4vr8\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721498 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmn4c\" (UniqueName: \"kubernetes.io/projected/1078985e-6c76-4495-b4f0-43cd7eeb645b-kube-api-access-hmn4c\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721523 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1078985e-6c76-4495-b4f0-43cd7eeb645b-config-volume\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721575 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721594 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721594 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a665b593-b6bf-4a32-a698-217aea9b6844-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.721687 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.721694 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:09.722310 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.721706 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:09.722310 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.721753 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:10.221734212 +0000 UTC m=+43.084741361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:09.722310 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.721910 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a8677844-41dc-4c2c-b82a-6ecf37554f9d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:09.723308 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.723288 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:09.723409 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.723345 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:10.223328395 +0000 UTC m=+43.086335544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:09.723409 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.723382 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7129d9f0-9ebf-4900-8622-8ea78d986a22-tmp\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.723996 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.723972 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.725816 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.725787 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.725816 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.725791 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.725816 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.725797 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.726034 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.725923 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.726034 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.725953 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7129d9f0-9ebf-4900-8622-8ea78d986a22-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.726190 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.726170 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.726312 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.726295 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-ca\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.726802 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.726784 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a665b593-b6bf-4a32-a698-217aea9b6844-hub\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.733189 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.733171 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4vr8\" (UniqueName: \"kubernetes.io/projected/2b38b5d7-8c17-4a31-86cd-52bbd191a88e-kube-api-access-p4vr8\") pod \"managed-serviceaccount-addon-agent-59dc676896-xd84z\" (UID: \"2b38b5d7-8c17-4a31-86cd-52bbd191a88e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.733511 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.733493 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.733571 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.733529 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbvs4\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:09.734780 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.734760 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9jz\" (UniqueName: \"kubernetes.io/projected/7129d9f0-9ebf-4900-8622-8ea78d986a22-kube-api-access-4t9jz\") pod \"klusterlet-addon-workmgr-78d68867cf-wv46z\" (UID: \"7129d9f0-9ebf-4900-8622-8ea78d986a22\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.735183 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.735153 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wljgp\" (UniqueName: \"kubernetes.io/projected/a665b593-b6bf-4a32-a698-217aea9b6844-kube-api-access-wljgp\") pod \"cluster-proxy-proxy-agent-695d9cd4ff-gt6v9\" (UID: \"a665b593-b6bf-4a32-a698-217aea9b6844\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.784675 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.784613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:09.820616 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.820592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" Jul 09 14:55:09.822862 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.822832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1078985e-6c76-4495-b4f0-43cd7eeb645b-tmp-dir\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.822994 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.822865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmcg\" (UniqueName: \"kubernetes.io/projected/2010273d-2add-4376-9b00-452a615ad1ce-kube-api-access-rrmcg\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.822994 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.822947 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmn4c\" (UniqueName: \"kubernetes.io/projected/1078985e-6c76-4495-b4f0-43cd7eeb645b-kube-api-access-hmn4c\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.822994 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.822987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1078985e-6c76-4495-b4f0-43cd7eeb645b-config-volume\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.823125 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.823016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.823125 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.823039 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2010273d-2add-4376-9b00-452a615ad1ce-cert\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.823274 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.823236 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:09.823274 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.823245 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1078985e-6c76-4495-b4f0-43cd7eeb645b-tmp-dir\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.823369 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:09.823298 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:10.323280187 +0000 UTC m=+43.186287362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:09.824150 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.823609 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1078985e-6c76-4495-b4f0-43cd7eeb645b-config-volume\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.829165 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.829145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2010273d-2add-4376-9b00-452a615ad1ce-cert\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.834803 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.834781 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmn4c\" (UniqueName: \"kubernetes.io/projected/1078985e-6c76-4495-b4f0-43cd7eeb645b-kube-api-access-hmn4c\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:09.835635 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.835611 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmcg\" (UniqueName: \"kubernetes.io/projected/2010273d-2add-4376-9b00-452a615ad1ce-kube-api-access-rrmcg\") pod \"ingress-canary-dgnf8\" (UID: \"2010273d-2add-4376-9b00-452a615ad1ce\") " pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:09.855555 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.855535 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:09.876180 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.876158 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:55:09.914993 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:09.914962 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dgnf8" Jul 09 14:55:10.050037 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.049843 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z"] Jul 09 14:55:10.054148 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.054117 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z"] Jul 09 14:55:10.054629 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:10.054604 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b38b5d7_8c17_4a31_86cd_52bbd191a88e.slice/crio-65c6a0d1ce6a7b68b9d358d4096d4d3cd6581afcc1c8d2cde7979317f873ae4f WatchSource:0}: Error finding container 65c6a0d1ce6a7b68b9d358d4096d4d3cd6581afcc1c8d2cde7979317f873ae4f: Status 404 returned error can't find the container with id 65c6a0d1ce6a7b68b9d358d4096d4d3cd6581afcc1c8d2cde7979317f873ae4f Jul 09 14:55:10.057279 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:10.057250 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7129d9f0_9ebf_4900_8622_8ea78d986a22.slice/crio-153f9378b229aa9f6b0df6672d09ff19c0299a38526b9d4554bbeb99a057676c WatchSource:0}: Error finding container 153f9378b229aa9f6b0df6672d09ff19c0299a38526b9d4554bbeb99a057676c: Status 404 returned error can't find the container with id 153f9378b229aa9f6b0df6672d09ff19c0299a38526b9d4554bbeb99a057676c Jul 09 14:55:10.086548 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.086520 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9"] Jul 09 14:55:10.089420 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:10.089397 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda665b593_b6bf_4a32_a698_217aea9b6844.slice/crio-6a8e1e4e2334513a25835bc82a40312198398f3ace9f9f8783a4b587b9d4a53d WatchSource:0}: Error finding container 6a8e1e4e2334513a25835bc82a40312198398f3ace9f9f8783a4b587b9d4a53d: Status 404 returned error can't find the container with id 6a8e1e4e2334513a25835bc82a40312198398f3ace9f9f8783a4b587b9d4a53d Jul 09 14:55:10.101600 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.101578 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dgnf8"] Jul 09 14:55:10.103850 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:10.103831 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2010273d_2add_4376_9b00_452a615ad1ce.slice/crio-acbce6f0887d038aa830fbfd29008d2f3035c72126689966a8b303ec0bbb46b8 WatchSource:0}: Error finding container acbce6f0887d038aa830fbfd29008d2f3035c72126689966a8b303ec0bbb46b8: Status 404 returned error can't find the container with id acbce6f0887d038aa830fbfd29008d2f3035c72126689966a8b303ec0bbb46b8 Jul 09 14:55:10.226052 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.226020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:10.226198 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.226064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:10.226198 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.226176 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:10.226293 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.226204 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:10.226293 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.226220 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:10.226293 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.226249 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:11.226228597 +0000 UTC m=+44.089235745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:10.226293 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.226269 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:11.226259618 +0000 UTC m=+44.089266768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:10.327056 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.326984 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:10.327189 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.327130 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:10.327224 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:10.327199 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:11.327182829 +0000 UTC m=+44.190189975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:10.618259 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.618189 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:10.618389 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.618189 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:10.618389 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.618189 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:10.622093 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.622003 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:55:10.622093 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.622011 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:55:10.622093 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.622039 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:55:10.622093 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.622098 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9zbh8\"" Jul 09 14:55:10.623536 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.623518 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:55:10.623623 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.623585 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c5h92\"" Jul 09 14:55:10.877577 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.877498 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" event={"ID":"2b38b5d7-8c17-4a31-86cd-52bbd191a88e","Type":"ContainerStarted","Data":"65c6a0d1ce6a7b68b9d358d4096d4d3cd6581afcc1c8d2cde7979317f873ae4f"} Jul 09 14:55:10.879024 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.878992 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dgnf8" event={"ID":"2010273d-2add-4376-9b00-452a615ad1ce","Type":"ContainerStarted","Data":"acbce6f0887d038aa830fbfd29008d2f3035c72126689966a8b303ec0bbb46b8"} Jul 09 14:55:10.881562 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.881537 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerStarted","Data":"6a8e1e4e2334513a25835bc82a40312198398f3ace9f9f8783a4b587b9d4a53d"} Jul 09 14:55:10.883718 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:10.883693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" event={"ID":"7129d9f0-9ebf-4900-8622-8ea78d986a22","Type":"ContainerStarted","Data":"153f9378b229aa9f6b0df6672d09ff19c0299a38526b9d4554bbeb99a057676c"} Jul 09 14:55:11.132386 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.132288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.219692 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.219617 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.235938 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.236080 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.236205 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.236260 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:13.236242929 +0000 UTC m=+46.099250079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.236626 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.236639 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:11.236713 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.236680 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:13.236665806 +0000 UTC m=+46.099672956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:11.286086 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.286004 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.337581 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.337030 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:11.337581 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.337186 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:11.337581 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:11.337246 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:13.337228826 +0000 UTC m=+46.200235977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:11.388952 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.388720 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.486830 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.486788 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.582285 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.582252 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.867536 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.867501 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:11.939596 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:11.939547 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:12.276435 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:12.276373 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:12.340985 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:12.340953 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:12.497090 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:12.497053 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:12.563683 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:12.563513 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:13.261434 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:13.261395 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:13.261449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.261571 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.261595 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.261608 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.261651 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:17.261632111 +0000 UTC m=+50.124639257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:13.261986 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.261671 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:17.261662366 +0000 UTC m=+50.124669515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:13.362677 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:13.362641 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:13.362859 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.362778 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:13.362859 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:13.362852 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:17.362832564 +0000 UTC m=+50.225839717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:17.298220 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:17.298182 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:17.298220 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:17.298225 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:17.298650 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.298330 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:17.298650 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.298400 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:25.298378443 +0000 UTC m=+58.161385590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:17.298650 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.298339 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:17.298650 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.298425 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:17.298650 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.298467 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:25.298454034 +0000 UTC m=+58.161461185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:17.399137 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:17.399100 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:17.399270 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.399249 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:17.399318 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:17.399307 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:25.399292012 +0000 UTC m=+58.262299158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:18.907723 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.907686 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dgnf8" event={"ID":"2010273d-2add-4376-9b00-452a615ad1ce","Type":"ContainerStarted","Data":"1396c5fc3735653f66886d1be698122c79767c537cb7c89b19108af65f91d9dd"} Jul 09 14:55:18.909824 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.909796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerStarted","Data":"3381e42f5e2c155ea3f4243ac37e6018b6e0ccd1a28138970f308a79b919234e"} Jul 09 14:55:18.909952 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.909837 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerStarted","Data":"f4b28b5443ff871d9a7375ef0f7e6e6421f138f8f23145b049c31cfbbbf3a34c"} Jul 09 14:55:18.909952 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.909857 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerStarted","Data":"8ff8f3cc771447f0ec454ad98d3e594703644a7fad7b705fe3d96a74c6115679"} Jul 09 14:55:18.911214 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.911195 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" event={"ID":"7129d9f0-9ebf-4900-8622-8ea78d986a22","Type":"ContainerStarted","Data":"8cf0c66d9423a431b74b6b88d0e76a7a0e83052199e26223e0c0df5fe7a45319"} Jul 09 14:55:18.911388 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.911366 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:18.912686 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.912666 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" event={"ID":"2b38b5d7-8c17-4a31-86cd-52bbd191a88e","Type":"ContainerStarted","Data":"20ec7d6e9f64ca00a66ba2ec11fd2fca16bdd75d7842a1c459595c5b35ec3ddc"} Jul 09 14:55:18.913187 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.913170 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:55:18.930476 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.930430 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dgnf8" podStartSLOduration=2.082133693 podStartE2EDuration="9.930419855s" totalImagesPullingTime="7.848286162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:18.92893831 +0000 UTC m=+51.791945496" watchObservedRunningTime="2026-07-09 14:55:18.930419855 +0000 UTC m=+51.793427001" Jul 09 14:55:18.949005 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.948984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:18.959518 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.959376 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" podStartSLOduration=17.062519387 podStartE2EDuration="24.959361011s" totalImagesPullingTime="7.896841624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:18.958470115 +0000 UTC m=+51.821477309" watchObservedRunningTime="2026-07-09 14:55:18.959361011 +0000 UTC m=+51.822368179" Jul 09 14:55:18.991425 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:18.991380 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" podStartSLOduration=17.081148346 podStartE2EDuration="24.991366076s" totalImagesPullingTime="7.91021773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:18.990616806 +0000 UTC m=+51.853623984" watchObservedRunningTime="2026-07-09 14:55:18.991366076 +0000 UTC m=+51.854373288" Jul 09 14:55:19.020919 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.020852 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" podStartSLOduration=17.158572192 podStartE2EDuration="25.020837922s" totalImagesPullingTime="7.86226573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:19.019798752 +0000 UTC m=+51.882805955" watchObservedRunningTime="2026-07-09 14:55:19.020837922 +0000 UTC m=+51.883845089" Jul 09 14:55:19.053301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.053268 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:19.143695 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.143662 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:19.245818 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.245791 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:19.345054 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.345016 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:19.447605 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:19.447576 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" limit=1024 Jul 09 14:55:25.356022 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:25.355974 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:25.356022 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:25.356024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:25.356487 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.356121 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:25.356487 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.356137 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:25.356487 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.356150 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:25.356487 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.356186 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:55:41.356169712 +0000 UTC m=+74.219176859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:25.356487 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.356202 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:55:41.356194464 +0000 UTC m=+74.219201610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:25.457103 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:25.457066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:25.457296 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.457208 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:25.457296 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:25.457264 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:55:41.457248802 +0000 UTC m=+74.320255949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:55:32.506226 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.506190 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:32.506644 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.506244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:55:32.510138 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.510118 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:55:32.510188 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.510160 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:55:32.517288 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:32.517271 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 14:55:32.517343 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:32.517332 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:56:36.517312954 +0000 UTC m=+129.380320101 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : secret "metrics-daemon-secret" not found Jul 09 14:55:32.520235 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.520211 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce2db69a-8723-42cc-8359-292bf2d8e76d-original-pull-secret\") pod \"global-pull-secret-syncer-jlq7p\" (UID: \"ce2db69a-8723-42cc-8359-292bf2d8e76d\") " pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:32.528440 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.528423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jlq7p" Jul 09 14:55:32.607363 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.607333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:32.611004 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.610982 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:55:32.621771 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.621750 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:55:32.630582 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.630561 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqxg9\" (UniqueName: \"kubernetes.io/projected/c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183-kube-api-access-rqxg9\") pod \"network-check-target-qg8dx\" (UID: \"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183\") " pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:32.650801 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.650784 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jlq7p"] Jul 09 14:55:32.653065 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:32.653040 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2db69a_8723_42cc_8359_292bf2d8e76d.slice/crio-6822026d614cda02830860c929d2000f48a79bcd047a905d46b2b3b20d8918b2 WatchSource:0}: Error finding container 6822026d614cda02830860c929d2000f48a79bcd047a905d46b2b3b20d8918b2: Status 404 returned error can't find the container with id 6822026d614cda02830860c929d2000f48a79bcd047a905d46b2b3b20d8918b2 Jul 09 14:55:32.843016 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.842938 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c5h92\"" Jul 09 14:55:32.849933 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.849917 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:32.957748 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.957718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jlq7p" event={"ID":"ce2db69a-8723-42cc-8359-292bf2d8e76d","Type":"ContainerStarted","Data":"6822026d614cda02830860c929d2000f48a79bcd047a905d46b2b3b20d8918b2"} Jul 09 14:55:32.970194 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:32.970175 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qg8dx"] Jul 09 14:55:32.972322 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:55:32.972298 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1aa04bf_4c12_4e9e_abc4_4e9fa93f6183.slice/crio-f361274d4dcb9cd6177a2bfb00b293a299d5aa6383673beef8992b79663b7783 WatchSource:0}: Error finding container f361274d4dcb9cd6177a2bfb00b293a299d5aa6383673beef8992b79663b7783: Status 404 returned error can't find the container with id f361274d4dcb9cd6177a2bfb00b293a299d5aa6383673beef8992b79663b7783 Jul 09 14:55:33.962641 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:33.962596 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qg8dx" event={"ID":"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183","Type":"ContainerStarted","Data":"f361274d4dcb9cd6177a2bfb00b293a299d5aa6383673beef8992b79663b7783"} Jul 09 14:55:36.883251 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:36.883225 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjdnn" Jul 09 14:55:37.977463 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:37.977425 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jlq7p" event={"ID":"ce2db69a-8723-42cc-8359-292bf2d8e76d","Type":"ContainerStarted","Data":"1974f13df268c0074a6a025badf47515c3bea039e011c995fcbe9c48c2c55cc0"} Jul 09 14:55:37.978933 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:37.978909 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qg8dx" event={"ID":"c1aa04bf-4c12-4e9e-abc4-4e9fa93f6183","Type":"ContainerStarted","Data":"3ab68967ab2a329a98a1665dfb226eb300f20ef393e940ce27d2c38c1a888628"} Jul 09 14:55:37.979040 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:37.979015 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:55:37.995380 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:37.995328 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jlq7p" podStartSLOduration=64.867936762 podStartE2EDuration="1m9.995311128s" totalImagesPullingTime="5.127374366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:37.994916634 +0000 UTC m=+70.857923798" watchObservedRunningTime="2026-07-09 14:55:37.995311128 +0000 UTC m=+70.858318296" Jul 09 14:55:38.025915 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:38.025833 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qg8dx" podStartSLOduration=65.22747743 podStartE2EDuration="1m10.025818905s" totalImagesPullingTime="4.798341475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:55:38.024851329 +0000 UTC m=+70.887858534" watchObservedRunningTime="2026-07-09 14:55:38.025818905 +0000 UTC m=+70.888826051" Jul 09 14:55:41.374658 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:41.374616 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:41.374665 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.374775 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.374778 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.374840 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:56:13.37482408 +0000 UTC m=+106.237831227 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.374852 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:55:41.375063 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.374925 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:56:13.374910868 +0000 UTC m=+106.237918015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:55:41.475369 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:55:41.475335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:55:41.475530 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.475476 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:55:41.475574 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:55:41.475547 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:56:13.475529961 +0000 UTC m=+106.338537108 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:56:08.983831 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:08.983800 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qg8dx" Jul 09 14:56:13.407374 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:13.407325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:56:13.407374 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:13.407381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:56:13.407971 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.407472 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:56:13.407971 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.407483 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c949fb998-9xj94: secret "image-registry-tls" not found Jul 09 14:56:13.407971 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.407508 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 14:56:13.407971 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.407544 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls podName:9474e207-9c1c-4ff1-9746-78e8bb0eaaa7 nodeName:}" failed. No retries permitted until 2026-07-09 14:57:17.407529244 +0000 UTC m=+170.270536394 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls") pod "image-registry-7c949fb998-9xj94" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7") : secret "image-registry-tls" not found Jul 09 14:56:13.407971 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.407585 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert podName:a8677844-41dc-4c2c-b82a-6ecf37554f9d nodeName:}" failed. No retries permitted until 2026-07-09 14:57:17.407567866 +0000 UTC m=+170.270575017 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-hqmnq" (UID: "a8677844-41dc-4c2c-b82a-6ecf37554f9d") : secret "networking-console-plugin-cert" not found Jul 09 14:56:13.508397 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:13.508368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:56:13.508552 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.508532 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:56:13.508619 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:13.508610 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls podName:1078985e-6c76-4495-b4f0-43cd7eeb645b nodeName:}" failed. No retries permitted until 2026-07-09 14:57:17.508588985 +0000 UTC m=+170.371596135 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls") pod "dns-default-zz5xw" (UID: "1078985e-6c76-4495-b4f0-43cd7eeb645b") : secret "dns-default-metrics-tls" not found Jul 09 14:56:36.592690 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:36.592646 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:56:36.593131 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:36.592765 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 14:56:36.593131 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:36.592824 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs podName:697d88da-9062-4432-aeb2-9cbb5a66299b nodeName:}" failed. No retries permitted until 2026-07-09 14:58:38.592810316 +0000 UTC m=+251.455817466 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs") pod "network-metrics-daemon-cz22s" (UID: "697d88da-9062-4432-aeb2-9cbb5a66299b") : secret "metrics-daemon-secret" not found Jul 09 14:56:55.316788 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.316754 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kgdlk"] Jul 09 14:56:55.318645 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.318629 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.325467 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.325446 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 14:56:55.325592 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.325446 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 14:56:55.328374 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.328358 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 14:56:55.328862 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.328848 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cq4x2\"" Jul 09 14:56:55.331308 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.331293 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 14:56:55.341146 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.341119 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kgdlk"] Jul 09 14:56:55.433566 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.433511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ed4f169-1b94-4415-a199-420f3d2858cd-data-volume\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.433566 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.433583 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.433809 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.433610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ed4f169-1b94-4415-a199-420f3d2858cd-crio-socket\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.433809 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.433644 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.433809 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.433701 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdb8n\" (UniqueName: \"kubernetes.io/projected/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-api-access-kdb8n\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534640 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534750 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534655 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ed4f169-1b94-4415-a199-420f3d2858cd-crio-socket\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534750 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534694 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534750 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534713 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdb8n\" (UniqueName: \"kubernetes.io/projected/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-api-access-kdb8n\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534750 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ed4f169-1b94-4415-a199-420f3d2858cd-data-volume\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534966 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:55.534752 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:56:55.534966 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:55.534813 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:56:55.534966 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.534821 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ed4f169-1b94-4415-a199-420f3d2858cd-crio-socket\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.534966 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:55.534833 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:56.034812689 +0000 UTC m=+148.897819845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:56:55.534966 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:55.534897 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:56.034863759 +0000 UTC m=+148.897870916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : configmap "kube-rbac-proxy" not found Jul 09 14:56:55.535124 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.535064 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ed4f169-1b94-4415-a199-420f3d2858cd-data-volume\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:55.545222 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:55.545192 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdb8n\" (UniqueName: \"kubernetes.io/projected/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-api-access-kdb8n\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:56.038567 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:56.038531 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:56.038742 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:56.038602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:56.038742 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:56.038673 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:56:56.038742 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:56.038697 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:56:56.038856 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:56.038755 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:57.038737077 +0000 UTC m=+149.901744228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:56:56.038856 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:56.038774 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:57.038764942 +0000 UTC m=+149.901772103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : configmap "kube-rbac-proxy" not found Jul 09 14:56:57.043831 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:57.043794 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:57.044271 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:57.043856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:57.044271 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:57.043948 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:56:57.044271 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:57.043974 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:56:57.044271 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:57.044001 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:59.043986756 +0000 UTC m=+151.906993904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : configmap "kube-rbac-proxy" not found Jul 09 14:56:57.044271 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:57.044028 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:56:59.044015727 +0000 UTC m=+151.907022874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:56:57.501349 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:57.501319 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4lnm_f1a9f981-930d-4fbd-a6a7-43e8bf453208/dns-node-resolver/0.log" Jul 09 14:56:58.100804 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:58.100776 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-glqpm_7e6cb45c-1646-42d3-b22f-89038504faf9/node-ca/0.log" Jul 09 14:56:58.701647 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:58.701606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dgnf8_2010273d-2add-4376-9b00-452a615ad1ce/serve-healthcheck-canary/0.log" Jul 09 14:56:59.058048 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:59.058016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:59.058245 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:56:59.058095 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:56:59.058245 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:59.058172 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:56:59.058245 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:59.058216 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:56:59.058389 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:59.058226 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:57:03.058211187 +0000 UTC m=+155.921218337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:56:59.058389 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:56:59.058301 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:57:03.058282096 +0000 UTC m=+155.921289249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : configmap "kube-rbac-proxy" not found Jul 09 14:57:03.089010 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:03.088976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:03.089478 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:03.089046 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:03.089478 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:03.089110 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:57:03.089478 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:03.089136 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:57:03.089478 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:03.089188 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:57:11.089167257 +0000 UTC m=+163.952174449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : configmap "kube-rbac-proxy" not found Jul 09 14:57:03.089478 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:03.089202 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:57:11.089196601 +0000 UTC m=+163.952203749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:57:11.149910 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:11.149853 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:11.150306 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:11.149946 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:11.150306 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:11.150032 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:57:11.150306 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:11.150084 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls podName:5ed4f169-1b94-4415-a199-420f3d2858cd nodeName:}" failed. No retries permitted until 2026-07-09 14:57:27.150070515 +0000 UTC m=+180.013077661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kgdlk" (UID: "5ed4f169-1b94-4415-a199-420f3d2858cd") : secret "insights-runtime-extractor-tls" not found Jul 09 14:57:11.150426 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:11.150404 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ed4f169-1b94-4415-a199-420f3d2858cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:12.494178 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:12.494137 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" podUID="a8677844-41dc-4c2c-b82a-6ecf37554f9d" Jul 09 14:57:12.529506 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:12.529468 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" Jul 09 14:57:12.609004 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:12.608962 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zz5xw" podUID="1078985e-6c76-4495-b4f0-43cd7eeb645b" Jul 09 14:57:13.242204 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:13.242174 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:57:13.242371 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:13.242177 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:13.242371 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:13.242177 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:13.634401 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:57:13.634306 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cz22s" podUID="697d88da-9062-4432-aeb2-9cbb5a66299b" Jul 09 14:57:17.495098 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.495064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:57:17.495464 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.495115 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:17.497420 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.497394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a8677844-41dc-4c2c-b82a-6ecf37554f9d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-hqmnq\" (UID: \"a8677844-41dc-4c2c-b82a-6ecf37554f9d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:57:17.497420 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.497408 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"image-registry-7c949fb998-9xj94\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:17.595470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.595414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:17.597650 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.597630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1078985e-6c76-4495-b4f0-43cd7eeb645b-metrics-tls\") pod \"dns-default-zz5xw\" (UID: \"1078985e-6c76-4495-b4f0-43cd7eeb645b\") " pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:17.747857 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.747780 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vdfrd\"" Jul 09 14:57:17.747857 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.747789 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z59h7\"" Jul 09 14:57:17.747857 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.747780 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rns85\"" Jul 09 14:57:17.753847 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.753830 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:17.753976 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.753907 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" Jul 09 14:57:17.753976 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.753932 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:17.903940 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:17.903913 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-hqmnq"] Jul 09 14:57:17.906483 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:57:17.906442 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8677844_41dc_4c2c_b82a_6ecf37554f9d.slice/crio-b0cc96e8a899550921b9af8e9f5acf04f23a32bef95bf60e4e361509e72a9693 WatchSource:0}: Error finding container b0cc96e8a899550921b9af8e9f5acf04f23a32bef95bf60e4e361509e72a9693: Status 404 returned error can't find the container with id b0cc96e8a899550921b9af8e9f5acf04f23a32bef95bf60e4e361509e72a9693 Jul 09 14:57:18.131718 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.131642 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zz5xw"] Jul 09 14:57:18.134417 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:57:18.134387 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1078985e_6c76_4495_b4f0_43cd7eeb645b.slice/crio-e51f91d34607f227fe9923bd17741a3ad397bca4d9dfbb3ade6388e1303cc872 WatchSource:0}: Error finding container e51f91d34607f227fe9923bd17741a3ad397bca4d9dfbb3ade6388e1303cc872: Status 404 returned error can't find the container with id e51f91d34607f227fe9923bd17741a3ad397bca4d9dfbb3ade6388e1303cc872 Jul 09 14:57:18.141335 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.141310 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:57:18.143241 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:57:18.143218 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9474e207_9c1c_4ff1_9746_78e8bb0eaaa7.slice/crio-364b278e9cdc8acc5fe2c988831751174e2f434977ac1dfb7d3a5ca3fe8cfc5e WatchSource:0}: Error finding container 364b278e9cdc8acc5fe2c988831751174e2f434977ac1dfb7d3a5ca3fe8cfc5e: Status 404 returned error can't find the container with id 364b278e9cdc8acc5fe2c988831751174e2f434977ac1dfb7d3a5ca3fe8cfc5e Jul 09 14:57:18.256548 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.256504 2558 generic.go:356] "Generic (PLEG): container finished" podID="7129d9f0-9ebf-4900-8622-8ea78d986a22" containerID="8cf0c66d9423a431b74b6b88d0e76a7a0e83052199e26223e0c0df5fe7a45319" exitCode=1 Jul 09 14:57:18.256663 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.256576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" event={"ID":"7129d9f0-9ebf-4900-8622-8ea78d986a22","Type":"ContainerDied","Data":"8cf0c66d9423a431b74b6b88d0e76a7a0e83052199e26223e0c0df5fe7a45319"} Jul 09 14:57:18.256990 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.256971 2558 scope.go:117] "RemoveContainer" containerID="8cf0c66d9423a431b74b6b88d0e76a7a0e83052199e26223e0c0df5fe7a45319" Jul 09 14:57:18.257901 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.257850 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" event={"ID":"a8677844-41dc-4c2c-b82a-6ecf37554f9d","Type":"ContainerStarted","Data":"b0cc96e8a899550921b9af8e9f5acf04f23a32bef95bf60e4e361509e72a9693"} Jul 09 14:57:18.259072 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.259043 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz5xw" event={"ID":"1078985e-6c76-4495-b4f0-43cd7eeb645b","Type":"ContainerStarted","Data":"e51f91d34607f227fe9923bd17741a3ad397bca4d9dfbb3ade6388e1303cc872"} Jul 09 14:57:18.260557 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.260531 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b38b5d7-8c17-4a31-86cd-52bbd191a88e" containerID="20ec7d6e9f64ca00a66ba2ec11fd2fca16bdd75d7842a1c459595c5b35ec3ddc" exitCode=255 Jul 09 14:57:18.260663 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.260594 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" event={"ID":"2b38b5d7-8c17-4a31-86cd-52bbd191a88e","Type":"ContainerDied","Data":"20ec7d6e9f64ca00a66ba2ec11fd2fca16bdd75d7842a1c459595c5b35ec3ddc"} Jul 09 14:57:18.260896 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.260864 2558 scope.go:117] "RemoveContainer" containerID="20ec7d6e9f64ca00a66ba2ec11fd2fca16bdd75d7842a1c459595c5b35ec3ddc" Jul 09 14:57:18.261802 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.261775 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" event={"ID":"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7","Type":"ContainerStarted","Data":"364b278e9cdc8acc5fe2c988831751174e2f434977ac1dfb7d3a5ca3fe8cfc5e"} Jul 09 14:57:18.911689 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:18.911561 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:57:19.266254 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.266221 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" event={"ID":"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7","Type":"ContainerStarted","Data":"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3"} Jul 09 14:57:19.266444 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.266352 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:19.267921 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.267898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" event={"ID":"7129d9f0-9ebf-4900-8622-8ea78d986a22","Type":"ContainerStarted","Data":"df5ba871da2c32643616023ce62489a8e0825a52942a5238c2350df855287cf8"} Jul 09 14:57:19.268029 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.267939 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:57:19.268613 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.268598 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d68867cf-wv46z" Jul 09 14:57:19.269616 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.269599 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-59dc676896-xd84z" event={"ID":"2b38b5d7-8c17-4a31-86cd-52bbd191a88e","Type":"ContainerStarted","Data":"b0d5eaae54f219d2ae21323fc9790795fb1c5765c44b722bedcf7ae825ce9709"} Jul 09 14:57:19.294926 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:19.294863 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" podStartSLOduration=171.294847528 podStartE2EDuration="2m51.294847528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:57:19.294401268 +0000 UTC m=+172.157408448" watchObservedRunningTime="2026-07-09 14:57:19.294847528 +0000 UTC m=+172.157854728" Jul 09 14:57:20.277188 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:20.277149 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" event={"ID":"a8677844-41dc-4c2c-b82a-6ecf37554f9d","Type":"ContainerStarted","Data":"bfea39ee69ba202409905c63cf3230bd046aaafb5b0fcaf9aadd336d233f6f97"} Jul 09 14:57:20.278927 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:20.278890 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz5xw" event={"ID":"1078985e-6c76-4495-b4f0-43cd7eeb645b","Type":"ContainerStarted","Data":"86b467ed3e6147f87b67b412dedb216b2258711a09da5f1a07f002f5f50edff1"} Jul 09 14:57:20.298025 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:20.297979 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-hqmnq" podStartSLOduration=160.918042148 podStartE2EDuration="2m42.297962326s" totalImagesPullingTime="1.379920178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:57:20.297754649 +0000 UTC m=+173.160761818" watchObservedRunningTime="2026-07-09 14:57:20.297962326 +0000 UTC m=+173.160969494" Jul 09 14:57:21.284171 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:21.284134 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zz5xw" event={"ID":"1078985e-6c76-4495-b4f0-43cd7eeb645b","Type":"ContainerStarted","Data":"43d538c4d1c84ade880fb466c40362825d18e9ff4bd58f05fd1b5eacd91bf3ee"} Jul 09 14:57:21.305060 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:21.305008 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zz5xw" podStartSLOduration=130.286316568 podStartE2EDuration="2m12.304989002s" totalImagesPullingTime="2.018672434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:57:21.30468017 +0000 UTC m=+174.167687341" watchObservedRunningTime="2026-07-09 14:57:21.304989002 +0000 UTC m=+174.167996171" Jul 09 14:57:22.288410 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:22.288376 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:27.167741 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:27.167689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:27.169923 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:27.169901 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ed4f169-1b94-4415-a199-420f3d2858cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kgdlk\" (UID: \"5ed4f169-1b94-4415-a199-420f3d2858cd\") " pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:27.426446 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:27.426372 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kgdlk" Jul 09 14:57:27.561646 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:27.561618 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kgdlk"] Jul 09 14:57:27.562847 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:57:27.562820 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed4f169_1b94_4415_a199_420f3d2858cd.slice/crio-721207939c8458d0855b98c3aed9e82cbff9e340f0cc9610f42043b0d1ef3150 WatchSource:0}: Error finding container 721207939c8458d0855b98c3aed9e82cbff9e340f0cc9610f42043b0d1ef3150: Status 404 returned error can't find the container with id 721207939c8458d0855b98c3aed9e82cbff9e340f0cc9610f42043b0d1ef3150 Jul 09 14:57:27.619012 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:27.618985 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:57:28.311013 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:28.310975 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kgdlk" event={"ID":"5ed4f169-1b94-4415-a199-420f3d2858cd","Type":"ContainerStarted","Data":"721207939c8458d0855b98c3aed9e82cbff9e340f0cc9610f42043b0d1ef3150"} Jul 09 14:57:31.323208 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:31.323170 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kgdlk" event={"ID":"5ed4f169-1b94-4415-a199-420f3d2858cd","Type":"ContainerStarted","Data":"217935d8110783a5109877501b10e625454f502893d3d6c1c48f6cd6dc79bf19"} Jul 09 14:57:32.293854 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:32.293823 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zz5xw" Jul 09 14:57:32.329506 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:32.329469 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kgdlk" event={"ID":"5ed4f169-1b94-4415-a199-420f3d2858cd","Type":"ContainerStarted","Data":"4dde53a086e162a82fbf382c86ab11c028043c1fe9a631257a1d4ad4b6f160dc"} Jul 09 14:57:35.341418 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:35.341384 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kgdlk" event={"ID":"5ed4f169-1b94-4415-a199-420f3d2858cd","Type":"ContainerStarted","Data":"e1e5d187ed3a82219cfb8cfef7231078b986fbe0abb11aa02f796b62039ea7db"} Jul 09 14:57:35.364835 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:35.364790 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kgdlk" podStartSLOduration=33.72021176 podStartE2EDuration="40.364772442s" totalImagesPullingTime="6.644560682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:56:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:57:35.364012929 +0000 UTC m=+188.227020098" watchObservedRunningTime="2026-07-09 14:57:35.364772442 +0000 UTC m=+188.227779612" Jul 09 14:57:37.031182 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.031151 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zrf2b"] Jul 09 14:57:37.034419 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.034399 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.037721 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.037700 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 14:57:37.039299 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039274 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 14:57:37.039470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039349 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 14:57:37.039470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039366 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 14:57:37.039470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039272 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 14:57:37.039470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039371 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 14:57:37.039470 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.039274 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rxjfj\"" Jul 09 14:57:37.147447 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-textfile\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147447 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147454 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtndt\" (UniqueName: \"kubernetes.io/projected/275ac97f-071b-4d99-b75a-5df17b478ae5-kube-api-access-dtndt\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147651 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147471 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-root\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147651 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147491 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147651 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147563 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-metrics-client-ca\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147651 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147597 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-wtmp\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147651 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147801 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147652 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-sys\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.147801 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.147704 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-tls\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248626 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248595 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248626 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248628 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-sys\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248796 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248649 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-tls\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248796 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-textfile\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248796 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248694 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtndt\" (UniqueName: \"kubernetes.io/projected/275ac97f-071b-4d99-b75a-5df17b478ae5-kube-api-access-dtndt\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248796 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248732 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-sys\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248969 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248808 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-root\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248969 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248845 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.248969 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-metrics-client-ca\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249106 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.248961 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-root\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249106 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.249011 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-wtmp\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249106 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.249074 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-textfile\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249239 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.249150 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-wtmp\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249322 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.249285 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-accelerators-collector-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.249450 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.249391 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/275ac97f-071b-4d99-b75a-5df17b478ae5-metrics-client-ca\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.251075 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.251048 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.251161 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.251138 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/275ac97f-071b-4d99-b75a-5df17b478ae5-node-exporter-tls\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.258640 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.258615 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtndt\" (UniqueName: \"kubernetes.io/projected/275ac97f-071b-4d99-b75a-5df17b478ae5-kube-api-access-dtndt\") pod \"node-exporter-zrf2b\" (UID: \"275ac97f-071b-4d99-b75a-5df17b478ae5\") " pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.342747 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.342674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zrf2b" Jul 09 14:57:37.354821 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:57:37.354797 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275ac97f_071b_4d99_b75a_5df17b478ae5.slice/crio-7e7e8007e29c4b321bd016a71dedf19822cb53eeb67f21f97d12c6db942f4ba5 WatchSource:0}: Error finding container 7e7e8007e29c4b321bd016a71dedf19822cb53eeb67f21f97d12c6db942f4ba5: Status 404 returned error can't find the container with id 7e7e8007e29c4b321bd016a71dedf19822cb53eeb67f21f97d12c6db942f4ba5 Jul 09 14:57:37.758931 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.758868 2558 patch_prober.go:28] interesting pod/image-registry-7c949fb998-9xj94 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 14:57:37.759105 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:37.758950 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:57:38.352627 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:38.352591 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrf2b" event={"ID":"275ac97f-071b-4d99-b75a-5df17b478ae5","Type":"ContainerStarted","Data":"7e7e8007e29c4b321bd016a71dedf19822cb53eeb67f21f97d12c6db942f4ba5"} Jul 09 14:57:39.359230 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:39.359195 2558 generic.go:356] "Generic (PLEG): container finished" podID="275ac97f-071b-4d99-b75a-5df17b478ae5" containerID="2457b21cab063285aa6f63dc62064e7ff203aeb969a925b659675449d406418f" exitCode=0 Jul 09 14:57:39.359654 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:39.359280 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrf2b" event={"ID":"275ac97f-071b-4d99-b75a-5df17b478ae5","Type":"ContainerDied","Data":"2457b21cab063285aa6f63dc62064e7ff203aeb969a925b659675449d406418f"} Jul 09 14:57:40.283114 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:40.283078 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:57:40.365348 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:40.365315 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrf2b" event={"ID":"275ac97f-071b-4d99-b75a-5df17b478ae5","Type":"ContainerStarted","Data":"00ad24ff37f2402019cd1aca8dd3af8e96e9cb2474bfd9bbe5d2e7cb384cf18e"} Jul 09 14:57:40.365739 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:40.365375 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zrf2b" event={"ID":"275ac97f-071b-4d99-b75a-5df17b478ae5","Type":"ContainerStarted","Data":"3923cdf372eb6513d3f3e8f150d76e9782c832082214cdfe6c3765bd45e92b7c"} Jul 09 14:57:40.400116 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:40.400066 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zrf2b" podStartSLOduration=2.17837444 podStartE2EDuration="3.400049407s" totalImagesPullingTime="1.155374843s" totalInitContainerRuntime="66.300124ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:57:40.399661946 +0000 UTC m=+193.262669125" watchObservedRunningTime="2026-07-09 14:57:40.400049407 +0000 UTC m=+193.263056575" Jul 09 14:57:44.726043 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:44.726011 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:57:49.878189 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:49.878150 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" podUID="a665b593-b6bf-4a32-a698-217aea9b6844" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 14:57:59.877737 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:57:59.877699 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" podUID="a665b593-b6bf-4a32-a698-217aea9b6844" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 14:58:09.745353 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.745304 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerName="registry" containerID="cri-o://6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3" gracePeriod=30 Jul 09 14:58:09.878084 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.878027 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" podUID="a665b593-b6bf-4a32-a698-217aea9b6844" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 14:58:09.878226 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.878130 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" Jul 09 14:58:09.878851 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.878811 2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3381e42f5e2c155ea3f4243ac37e6018b6e0ccd1a28138970f308a79b919234e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 09 14:58:09.878919 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.878899 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" podUID="a665b593-b6bf-4a32-a698-217aea9b6844" containerName="service-proxy" containerID="cri-o://3381e42f5e2c155ea3f4243ac37e6018b6e0ccd1a28138970f308a79b919234e" gracePeriod=30 Jul 09 14:58:09.997112 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:09.997093 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:58:10.098301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098271 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098301 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098301 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098499 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098322 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098499 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098360 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098499 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098391 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098499 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098428 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098499 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098483 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098760 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098506 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbvs4\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4\") pod \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\" (UID: \"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7\") " Jul 09 14:58:10.098760 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.098708 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:58:10.099267 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.099187 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:58:10.100721 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.100695 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:58:10.100827 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.100779 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:58:10.100827 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.100801 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:58:10.101016 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.100995 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4" (OuterVolumeSpecName: "kube-api-access-lbvs4") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "kube-api-access-lbvs4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:58:10.101150 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.101127 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:58:10.109292 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.109271 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" (UID: "9474e207-9c1c-4ff1-9746-78e8bb0eaaa7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:58:10.199503 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199462 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbvs4\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-kube-api-access-lbvs4\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199503 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199500 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-trusted-ca\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199503 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199513 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-tls\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199698 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199525 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-ca-trust-extracted\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199698 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199537 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-image-registry-private-configuration\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199698 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199549 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-installation-pull-secrets\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199698 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199561 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-bound-sa-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.199698 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.199574 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7-registry-certificates\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 14:58:10.460529 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.460495 2558 generic.go:356] "Generic (PLEG): container finished" podID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerID="6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3" exitCode=0 Jul 09 14:58:10.460699 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.460541 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" Jul 09 14:58:10.460699 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.460576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" event={"ID":"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7","Type":"ContainerDied","Data":"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3"} Jul 09 14:58:10.460699 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.460634 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c949fb998-9xj94" event={"ID":"9474e207-9c1c-4ff1-9746-78e8bb0eaaa7","Type":"ContainerDied","Data":"364b278e9cdc8acc5fe2c988831751174e2f434977ac1dfb7d3a5ca3fe8cfc5e"} Jul 09 14:58:10.460699 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.460655 2558 scope.go:117] "RemoveContainer" containerID="6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3" Jul 09 14:58:10.463061 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.463035 2558 generic.go:356] "Generic (PLEG): container finished" podID="a665b593-b6bf-4a32-a698-217aea9b6844" containerID="3381e42f5e2c155ea3f4243ac37e6018b6e0ccd1a28138970f308a79b919234e" exitCode=2 Jul 09 14:58:10.463198 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.463075 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerDied","Data":"3381e42f5e2c155ea3f4243ac37e6018b6e0ccd1a28138970f308a79b919234e"} Jul 09 14:58:10.463198 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.463101 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695d9cd4ff-gt6v9" event={"ID":"a665b593-b6bf-4a32-a698-217aea9b6844","Type":"ContainerStarted","Data":"0e795b507efcf9dc79d232cbcd0ae0d7ca0504d119016619f1ce794daa16ddf1"} Jul 09 14:58:10.469171 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.469157 2558 scope.go:117] "RemoveContainer" containerID="6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3" Jul 09 14:58:10.469421 ip-10-0-142-231 kubenswrapper[2558]: E0709 14:58:10.469401 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3\": container with ID starting with 6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3 not found: ID does not exist" containerID="6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3" Jul 09 14:58:10.469488 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.469428 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3"} err="failed to get container status \"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3\": rpc error: code = NotFound desc = could not find container \"6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3\": container with ID starting with 6b7529f27211baa5a05905a78324795373db0680a46472b0e358f5529c4c89a3 not found: ID does not exist" Jul 09 14:58:10.503689 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.503664 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:58:10.509693 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:10.509675 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c949fb998-9xj94"] Jul 09 14:58:11.621694 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:11.621663 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" path="/var/lib/kubelet/pods/9474e207-9c1c-4ff1-9746-78e8bb0eaaa7/volumes" Jul 09 14:58:15.619299 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:15.619271 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz5xw_1078985e-6c76-4495-b4f0-43cd7eeb645b/dns/0.log" Jul 09 14:58:15.627602 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:15.627580 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz5xw_1078985e-6c76-4495-b4f0-43cd7eeb645b/kube-rbac-proxy/0.log" Jul 09 14:58:16.165774 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:16.165744 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4lnm_f1a9f981-930d-4fbd-a6a7-43e8bf453208/dns-node-resolver/0.log" Jul 09 14:58:38.684961 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:38.684925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:58:38.687361 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:38.687337 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697d88da-9062-4432-aeb2-9cbb5a66299b-metrics-certs\") pod \"network-metrics-daemon-cz22s\" (UID: \"697d88da-9062-4432-aeb2-9cbb5a66299b\") " pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:58:38.722727 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:38.722704 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9zbh8\"" Jul 09 14:58:38.730346 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:38.730328 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cz22s" Jul 09 14:58:38.855906 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:38.855792 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cz22s"] Jul 09 14:58:38.858530 ip-10-0-142-231 kubenswrapper[2558]: W0709 14:58:38.858502 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697d88da_9062_4432_aeb2_9cbb5a66299b.slice/crio-5864601c353b8746a2da4b8f1257cfbf6bbd686c4251aca4dd1b99589f3e376d WatchSource:0}: Error finding container 5864601c353b8746a2da4b8f1257cfbf6bbd686c4251aca4dd1b99589f3e376d: Status 404 returned error can't find the container with id 5864601c353b8746a2da4b8f1257cfbf6bbd686c4251aca4dd1b99589f3e376d Jul 09 14:58:39.563774 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:39.563733 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cz22s" event={"ID":"697d88da-9062-4432-aeb2-9cbb5a66299b","Type":"ContainerStarted","Data":"5864601c353b8746a2da4b8f1257cfbf6bbd686c4251aca4dd1b99589f3e376d"} Jul 09 14:58:40.569286 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:40.569251 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cz22s" event={"ID":"697d88da-9062-4432-aeb2-9cbb5a66299b","Type":"ContainerStarted","Data":"77949e9a646641a88eb08cedde8c9aaa99118732e0f6cc294db057b47735c3e0"} Jul 09 14:58:40.569675 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:40.569302 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cz22s" event={"ID":"697d88da-9062-4432-aeb2-9cbb5a66299b","Type":"ContainerStarted","Data":"b3a6762670da8fc4079e286b9afe13b9df8c0b910e1d15f28165760d810ca1ff"} Jul 09 14:58:40.590752 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:58:40.590702 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cz22s" podStartSLOduration=251.606127114 podStartE2EDuration="4m12.590687494s" totalImagesPullingTime="984.56038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:58:40.589118882 +0000 UTC m=+253.452126073" watchObservedRunningTime="2026-07-09 14:58:40.590687494 +0000 UTC m=+253.453694664" Jul 09 14:59:27.549870 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:59:27.549847 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:59:27.553907 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:59:27.553867 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 14:59:27.562114 ip-10-0-142-231 kubenswrapper[2558]: I0709 14:59:27.562095 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 15:00:13.418705 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.418673 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-csmhj"] Jul 09 15:00:13.421028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.418953 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerName="registry" Jul 09 15:00:13.421028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.418966 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerName="registry" Jul 09 15:00:13.421028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.419006 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9474e207-9c1c-4ff1-9746-78e8bb0eaaa7" containerName="registry" Jul 09 15:00:13.421921 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.421907 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.424885 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.424854 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 15:00:13.426158 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.426140 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nps6b\"" Jul 09 15:00:13.426251 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.426171 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 15:00:13.430101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.430079 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-csmhj"] Jul 09 15:00:13.615797 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.615763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.615973 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.615820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdvtb\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-kube-api-access-qdvtb\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.716782 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.716760 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.716947 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.716808 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdvtb\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-kube-api-access-qdvtb\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.725942 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.725921 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.726036 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.726003 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdvtb\" (UniqueName: \"kubernetes.io/projected/12d6a588-e4e6-4f7f-b2f8-32a249795f49-kube-api-access-qdvtb\") pod \"cert-manager-cainjector-559cdd8c87-csmhj\" (UID: \"12d6a588-e4e6-4f7f-b2f8-32a249795f49\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.730796 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.730777 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" Jul 09 15:00:13.852136 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.851935 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-csmhj"] Jul 09 15:00:13.854343 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:00:13.854317 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12d6a588_e4e6_4f7f_b2f8_32a249795f49.slice/crio-751d669832f422819d1a34abdb645014685511a72f2349d7c3ca0dff7fd4c84b WatchSource:0}: Error finding container 751d669832f422819d1a34abdb645014685511a72f2349d7c3ca0dff7fd4c84b: Status 404 returned error can't find the container with id 751d669832f422819d1a34abdb645014685511a72f2349d7c3ca0dff7fd4c84b Jul 09 15:00:13.856732 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:13.856718 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:00:14.859060 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:14.859013 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" event={"ID":"12d6a588-e4e6-4f7f-b2f8-32a249795f49","Type":"ContainerStarted","Data":"751d669832f422819d1a34abdb645014685511a72f2349d7c3ca0dff7fd4c84b"} Jul 09 15:00:17.871413 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:17.871381 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" event={"ID":"12d6a588-e4e6-4f7f-b2f8-32a249795f49","Type":"ContainerStarted","Data":"958ebc2c5a1ac9f6828de16c877c64bcf8de2a4884b010514641ac0d49b3cbd4"} Jul 09 15:00:17.895611 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:17.895567 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-csmhj" podStartSLOduration=1.567639142 podStartE2EDuration="4.89555025s" totalImagesPullingTime="3.327911108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:00:17.894574724 +0000 UTC m=+350.757581893" watchObservedRunningTime="2026-07-09 15:00:17.89555025 +0000 UTC m=+350.758557419" Jul 09 15:00:44.330984 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.330938 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5"] Jul 09 15:00:44.336112 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.336079 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.341958 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.341922 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p9w5l\"" Jul 09 15:00:44.342518 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.342488 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 15:00:44.342656 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.342535 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 15:00:44.342656 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.342550 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 15:00:44.343359 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.343331 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 15:00:44.358953 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.358896 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5"] Jul 09 15:00:44.431382 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.431329 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.431382 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.431380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.431681 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.431496 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4pc5\" (UniqueName: \"kubernetes.io/projected/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-kube-api-access-s4pc5\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.431681 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.431556 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-manifests\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.431681 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.431587 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-tmp\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.532683 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.532636 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.532683 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.532684 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.533028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.532751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4pc5\" (UniqueName: \"kubernetes.io/projected/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-kube-api-access-s4pc5\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.533028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.532793 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-manifests\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.533028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.532925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-tmp\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.533336 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.533300 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-manifests\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.535740 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.535699 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-tmp\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.535950 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.535828 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.536028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.536006 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.546340 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.546299 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4pc5\" (UniqueName: \"kubernetes.io/projected/d7f318c3-af8d-4f18-b1e6-c6fbebc86184-kube-api-access-s4pc5\") pod \"opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5\" (UID: \"d7f318c3-af8d-4f18-b1e6-c6fbebc86184\") " pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.647754 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.647631 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:44.822844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.822768 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5"] Jul 09 15:00:44.826377 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:00:44.826345 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f318c3_af8d_4f18_b1e6_c6fbebc86184.slice/crio-1af0de47c19482f72e03822e0a1d98add99ff580799930447671f73668a1ea97 WatchSource:0}: Error finding container 1af0de47c19482f72e03822e0a1d98add99ff580799930447671f73668a1ea97: Status 404 returned error can't find the container with id 1af0de47c19482f72e03822e0a1d98add99ff580799930447671f73668a1ea97 Jul 09 15:00:44.965044 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:44.965003 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" event={"ID":"d7f318c3-af8d-4f18-b1e6-c6fbebc86184","Type":"ContainerStarted","Data":"1af0de47c19482f72e03822e0a1d98add99ff580799930447671f73668a1ea97"} Jul 09 15:00:47.981072 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:47.981024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" event={"ID":"d7f318c3-af8d-4f18-b1e6-c6fbebc86184","Type":"ContainerStarted","Data":"a00594e6cde1f99dd413c2087d53beabd5ae09003127ccc31392f3df28fa4b27"} Jul 09 15:00:49.990766 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:49.990720 2558 generic.go:356] "Generic (PLEG): container finished" podID="d7f318c3-af8d-4f18-b1e6-c6fbebc86184" containerID="a00594e6cde1f99dd413c2087d53beabd5ae09003127ccc31392f3df28fa4b27" exitCode=0 Jul 09 15:00:49.991262 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:49.990794 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" event={"ID":"d7f318c3-af8d-4f18-b1e6-c6fbebc86184","Type":"ContainerDied","Data":"a00594e6cde1f99dd413c2087d53beabd5ae09003127ccc31392f3df28fa4b27"} Jul 09 15:00:50.997581 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:50.997523 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" event={"ID":"d7f318c3-af8d-4f18-b1e6-c6fbebc86184","Type":"ContainerStarted","Data":"489947c9dc8956d05048bc029d95ab8f1616e8cdebb6753cda530a544627c652"} Jul 09 15:00:50.998039 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:50.997783 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:00:51.037663 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:00:51.037599 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" podStartSLOduration=2.457514576 podStartE2EDuration="7.037577853s" totalImagesPullingTime="3.356179972s" totalInitContainerRuntime="1.223883305s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:00:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:00:51.035362666 +0000 UTC m=+383.898369873" watchObservedRunningTime="2026-07-09 15:00:51.037577853 +0000 UTC m=+383.900585021" Jul 09 15:01:02.004634 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:02.004603 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5" Jul 09 15:01:04.172899 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.172850 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk"] Jul 09 15:01:04.180855 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.180834 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.183939 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.183920 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 15:01:04.184047 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.183960 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 15:01:04.185486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.185464 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4mlg7\"" Jul 09 15:01:04.185486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.185480 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 15:01:04.185646 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.185512 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 15:01:04.195979 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.195953 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk"] Jul 09 15:01:04.277679 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.277643 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmn2x"] Jul 09 15:01:04.281041 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.281020 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.286562 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.286541 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 15:01:04.293004 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.292987 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vpbfh\"" Jul 09 15:01:04.294360 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.294342 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/514dc168-0970-43d3-a336-91e850eb700c-tmp\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.294446 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.294400 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/514dc168-0970-43d3-a336-91e850eb700c-tls-certs\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.294483 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.294449 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6mgv\" (UniqueName: \"kubernetes.io/projected/514dc168-0970-43d3-a336-91e850eb700c-kube-api-access-t6mgv\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.296613 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.296595 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmn2x"] Jul 09 15:01:04.394991 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.394960 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/514dc168-0970-43d3-a336-91e850eb700c-tls-certs\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.395163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.395001 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6mgv\" (UniqueName: \"kubernetes.io/projected/514dc168-0970-43d3-a336-91e850eb700c-kube-api-access-t6mgv\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.395163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.395021 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/514dc168-0970-43d3-a336-91e850eb700c-tmp\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.395163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.395079 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.395163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.395122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68xhg\" (UniqueName: \"kubernetes.io/projected/838195b4-8bfc-485c-bb95-0b973e5e8864-kube-api-access-68xhg\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.397395 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.397363 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/514dc168-0970-43d3-a336-91e850eb700c-tmp\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.397617 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.397602 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/514dc168-0970-43d3-a336-91e850eb700c-tls-certs\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.422388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.422357 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6mgv\" (UniqueName: \"kubernetes.io/projected/514dc168-0970-43d3-a336-91e850eb700c-kube-api-access-t6mgv\") pod \"kube-auth-proxy-99cb46dbc-m4tlk\" (UID: \"514dc168-0970-43d3-a336-91e850eb700c\") " pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.490240 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.490210 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" Jul 09 15:01:04.496416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.496398 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.496488 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.496424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68xhg\" (UniqueName: \"kubernetes.io/projected/838195b4-8bfc-485c-bb95-0b973e5e8864-kube-api-access-68xhg\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.496562 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:01:04.496543 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 15:01:04.496622 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:01:04.496613 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert podName:838195b4-8bfc-485c-bb95-0b973e5e8864 nodeName:}" failed. No retries permitted until 2026-07-09 15:01:04.996596036 +0000 UTC m=+397.859603183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert") pod "odh-model-controller-7998c67dc4-rmn2x" (UID: "838195b4-8bfc-485c-bb95-0b973e5e8864") : secret "odh-model-controller-webhook-cert" not found Jul 09 15:01:04.508096 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.508075 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68xhg\" (UniqueName: \"kubernetes.io/projected/838195b4-8bfc-485c-bb95-0b973e5e8864-kube-api-access-68xhg\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:04.620121 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:04.620083 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk"] Jul 09 15:01:04.622752 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:04.622725 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514dc168_0970_43d3_a336_91e850eb700c.slice/crio-0a705e54eee49874ac1b58483327241b5f61b6e3c72c65e0c961a52ac6f970bb WatchSource:0}: Error finding container 0a705e54eee49874ac1b58483327241b5f61b6e3c72c65e0c961a52ac6f970bb: Status 404 returned error can't find the container with id 0a705e54eee49874ac1b58483327241b5f61b6e3c72c65e0c961a52ac6f970bb Jul 09 15:01:05.001161 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.001124 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:05.003441 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.003423 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/838195b4-8bfc-485c-bb95-0b973e5e8864-cert\") pod \"odh-model-controller-7998c67dc4-rmn2x\" (UID: \"838195b4-8bfc-485c-bb95-0b973e5e8864\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:05.047908 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.047804 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" event={"ID":"514dc168-0970-43d3-a336-91e850eb700c","Type":"ContainerStarted","Data":"0a705e54eee49874ac1b58483327241b5f61b6e3c72c65e0c961a52ac6f970bb"} Jul 09 15:01:05.190604 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.190567 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:05.252170 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.252080 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lpc4j"] Jul 09 15:01:05.255519 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.255497 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.258382 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.258360 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 15:01:05.267993 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.267970 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lpc4j"] Jul 09 15:01:05.322635 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.322614 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmn2x"] Jul 09 15:01:05.324385 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:05.324361 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838195b4_8bfc_485c_bb95_0b973e5e8864.slice/crio-205e4dc959a4ebb16d69743f69d016debdc17fb3398d655569b18c2e95c04eee WatchSource:0}: Error finding container 205e4dc959a4ebb16d69743f69d016debdc17fb3398d655569b18c2e95c04eee: Status 404 returned error can't find the container with id 205e4dc959a4ebb16d69743f69d016debdc17fb3398d655569b18c2e95c04eee Jul 09 15:01:05.404562 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.404522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fn7b\" (UniqueName: \"kubernetes.io/projected/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-kube-api-access-8fn7b\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.404731 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.404602 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.467809 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.467767 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:05.470847 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.470826 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.474434 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.474415 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 15:01:05.474538 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.474421 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 15:01:05.474600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.474547 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 15:01:05.474684 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.474607 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-whkfl\"" Jul 09 15:01:05.474817 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.474802 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 15:01:05.485446 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.485426 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:05.505285 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.505217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fn7b\" (UniqueName: \"kubernetes.io/projected/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-kube-api-access-8fn7b\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.505285 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.505259 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.505418 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:01:05.505362 2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 15:01:05.505454 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:01:05.505418 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs podName:4e609a77-2a0d-4292-b1cf-438eb31fc0d1 nodeName:}" failed. No retries permitted until 2026-07-09 15:01:06.005400377 +0000 UTC m=+398.868407541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs") pod "model-serving-api-54d89bd5fc-lpc4j" (UID: "4e609a77-2a0d-4292-b1cf-438eb31fc0d1") : secret "model-serving-api-tls" not found Jul 09 15:01:05.525049 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.525011 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fn7b\" (UniqueName: \"kubernetes.io/projected/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-kube-api-access-8fn7b\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:05.606027 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.605987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606028 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5ns8\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606104 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606124 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606389 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606229 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.606389 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.606258 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707211 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707321 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707344 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5ns8\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.707450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.707399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.708167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.708141 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.710083 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.710061 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.710197 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.710177 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.710260 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.710238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.710478 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.710457 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.718527 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.718460 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.718914 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.718848 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5ns8\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8\") pod \"istiod-openshift-gateway-7574459656-wz72p\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.780574 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.780499 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:05.961986 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:05.961959 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:05.963719 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:05.963689 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a8037f_0398_4c2d_ad9f_10e34592b26f.slice/crio-bc0c58203473bd342a4fe2bce118d1a6039986dd1280d06b37940b8cf02b8e8d WatchSource:0}: Error finding container bc0c58203473bd342a4fe2bce118d1a6039986dd1280d06b37940b8cf02b8e8d: Status 404 returned error can't find the container with id bc0c58203473bd342a4fe2bce118d1a6039986dd1280d06b37940b8cf02b8e8d Jul 09 15:01:06.010578 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.010536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:06.013110 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.013086 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e609a77-2a0d-4292-b1cf-438eb31fc0d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-lpc4j\" (UID: \"4e609a77-2a0d-4292-b1cf-438eb31fc0d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:06.053250 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.053123 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" event={"ID":"83a8037f-0398-4c2d-ad9f-10e34592b26f","Type":"ContainerStarted","Data":"bc0c58203473bd342a4fe2bce118d1a6039986dd1280d06b37940b8cf02b8e8d"} Jul 09 15:01:06.054481 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.054441 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" event={"ID":"838195b4-8bfc-485c-bb95-0b973e5e8864","Type":"ContainerStarted","Data":"205e4dc959a4ebb16d69743f69d016debdc17fb3398d655569b18c2e95c04eee"} Jul 09 15:01:06.166037 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.165994 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:06.298676 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:06.298641 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lpc4j"] Jul 09 15:01:06.300065 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:06.300038 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e609a77_2a0d_4292_b1cf_438eb31fc0d1.slice/crio-8080227d9312cf482412adb1c3c48a9a48f67c54a435ce092feb954294297891 WatchSource:0}: Error finding container 8080227d9312cf482412adb1c3c48a9a48f67c54a435ce092feb954294297891: Status 404 returned error can't find the container with id 8080227d9312cf482412adb1c3c48a9a48f67c54a435ce092feb954294297891 Jul 09 15:01:07.059800 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:07.059759 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" event={"ID":"4e609a77-2a0d-4292-b1cf-438eb31fc0d1","Type":"ContainerStarted","Data":"8080227d9312cf482412adb1c3c48a9a48f67c54a435ce092feb954294297891"} Jul 09 15:01:10.079889 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:10.079815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" event={"ID":"4e609a77-2a0d-4292-b1cf-438eb31fc0d1","Type":"ContainerStarted","Data":"db0811220b011ca03b380bb825b6c70551f397bc281ab46b659e23589b219b32"} Jul 09 15:01:10.080474 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:10.079943 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:10.081998 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:10.081957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" event={"ID":"514dc168-0970-43d3-a336-91e850eb700c","Type":"ContainerStarted","Data":"a09ca3260d6cac4ef1b7c23c67e0480fb88e1d51102de7bec752585b02ada642"} Jul 09 15:01:10.113357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:10.113276 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" podStartSLOduration=2.254922263 podStartE2EDuration="5.113246193s" totalImagesPullingTime="2.85832393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:10.110636877 +0000 UTC m=+402.973644047" watchObservedRunningTime="2026-07-09 15:01:10.113246193 +0000 UTC m=+402.976253341" Jul 09 15:01:10.143417 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:10.143350 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-99cb46dbc-m4tlk" podStartSLOduration=1.598875779 podStartE2EDuration="6.143324977s" totalImagesPullingTime="4.544449198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:10.141721564 +0000 UTC m=+403.004728734" watchObservedRunningTime="2026-07-09 15:01:10.143324977 +0000 UTC m=+403.006332198" Jul 09 15:01:11.674558 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:11.674505 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:01:11.675051 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:11.674632 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:01:12.094105 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.094062 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" event={"ID":"83a8037f-0398-4c2d-ad9f-10e34592b26f","Type":"ContainerStarted","Data":"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38"} Jul 09 15:01:12.094406 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.094329 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:12.096128 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.096014 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" event={"ID":"838195b4-8bfc-485c-bb95-0b973e5e8864","Type":"ContainerStarted","Data":"c784ee50d35a2fe24b18b69a00a0235327bcdbfa3def7f46d9e756ed330c261b"} Jul 09 15:01:12.096279 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.096174 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:12.096450 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.096410 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-wz72p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 15:01:12.096557 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.096486 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 15:01:12.152606 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.152536 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" podStartSLOduration=1.444239123 podStartE2EDuration="7.152512601s" totalImagesPullingTime="5.708273478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:12.149736712 +0000 UTC m=+405.012743892" watchObservedRunningTime="2026-07-09 15:01:12.152512601 +0000 UTC m=+405.015519769" Jul 09 15:01:12.208276 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:12.208202 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" podStartSLOduration=1.9100305450000001 podStartE2EDuration="8.208178792s" totalImagesPullingTime="6.298148247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:12.207011343 +0000 UTC m=+405.070018531" watchObservedRunningTime="2026-07-09 15:01:12.208178792 +0000 UTC m=+405.071185961" Jul 09 15:01:13.101506 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.101456 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-wz72p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 15:01:13.101988 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.101535 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 15:01:13.689038 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.688993 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg"] Jul 09 15:01:13.692771 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.692743 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.696028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.695998 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2cs76\"" Jul 09 15:01:13.706950 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.706923 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg"] Jul 09 15:01:13.786840 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.786799 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftrvb\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-kube-api-access-ftrvb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.786840 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.786845 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787131 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.786903 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787131 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.786930 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787131 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.786995 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787131 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.787041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787131 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.787062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787369 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.787120 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.787369 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.787173 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887614 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887563 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887642 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887677 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887725 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887760 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftrvb\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-kube-api-access-ftrvb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.887832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888204 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887837 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888204 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.887864 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888306 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.888259 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888306 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.888266 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888633 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.888605 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888736 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.888646 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.888964 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.888946 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.890335 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.890312 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.890476 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.890459 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.904545 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.904510 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftrvb\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-kube-api-access-ftrvb\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:13.904705 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:13.904684 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8e766ed-bc9a-494f-af79-b469aa2ad1a4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mt8xg\" (UID: \"f8e766ed-bc9a-494f-af79-b469aa2ad1a4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:14.003239 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.003201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:14.168935 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.168902 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg"] Jul 09 15:01:14.171118 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:14.171087 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e766ed_bc9a_494f_af79_b469aa2ad1a4.slice/crio-5e034f65c848ba2fae419d380060427bd0f93790a3a52de570fc3696c6846215 WatchSource:0}: Error finding container 5e034f65c848ba2fae419d380060427bd0f93790a3a52de570fc3696c6846215: Status 404 returned error can't find the container with id 5e034f65c848ba2fae419d380060427bd0f93790a3a52de570fc3696c6846215 Jul 09 15:01:14.945761 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.945711 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7"] Jul 09 15:01:14.951058 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.950983 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n"] Jul 09 15:01:14.952234 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.951142 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:14.955133 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.955107 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 15:01:14.955347 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.955247 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-whccr\"" Jul 09 15:01:14.955792 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.955767 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:14.959033 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.959007 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 15:01:14.959568 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.959542 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lqcjs\"" Jul 09 15:01:14.967863 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.967836 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7"] Jul 09 15:01:14.985489 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:14.985402 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n"] Jul 09 15:01:15.098804 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.098724 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a42114fe-c4a0-4d5a-8e1d-192845f031c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.098804 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.098780 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87v56\" (UniqueName: \"kubernetes.io/projected/a42114fe-c4a0-4d5a-8e1d-192845f031c8-kube-api-access-87v56\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.099047 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.098866 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.099047 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.098930 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t429p\" (UniqueName: \"kubernetes.io/projected/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-kube-api-access-t429p\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.111755 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.111716 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" event={"ID":"f8e766ed-bc9a-494f-af79-b469aa2ad1a4","Type":"ContainerStarted","Data":"5e034f65c848ba2fae419d380060427bd0f93790a3a52de570fc3696c6846215"} Jul 09 15:01:15.199496 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.199463 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.199973 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.199507 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t429p\" (UniqueName: \"kubernetes.io/projected/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-kube-api-access-t429p\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.199973 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.199556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a42114fe-c4a0-4d5a-8e1d-192845f031c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.199973 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.199610 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87v56\" (UniqueName: \"kubernetes.io/projected/a42114fe-c4a0-4d5a-8e1d-192845f031c8-kube-api-access-87v56\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.202475 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.202447 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a42114fe-c4a0-4d5a-8e1d-192845f031c8-cert\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.202584 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.202474 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.209171 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.209145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87v56\" (UniqueName: \"kubernetes.io/projected/a42114fe-c4a0-4d5a-8e1d-192845f031c8-kube-api-access-87v56\") pod \"kserve-controller-manager-d6f4b65b8-dfzr7\" (UID: \"a42114fe-c4a0-4d5a-8e1d-192845f031c8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.210220 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.210195 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t429p\" (UniqueName: \"kubernetes.io/projected/a7c673c8-b24c-4d4f-9ef7-1e91c41546af-kube-api-access-t429p\") pod \"llmisvc-controller-manager-777c6bd7d5-6h24n\" (UID: \"a7c673c8-b24c-4d4f-9ef7-1e91c41546af\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.267084 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.267040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:15.275279 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.275258 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:15.444632 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:15.444598 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42114fe_c4a0_4d5a_8e1d_192845f031c8.slice/crio-0834d01ae762db02e31cf7e45772ffc4f81952d085e04f05400f6f5fbdbe5ff6 WatchSource:0}: Error finding container 0834d01ae762db02e31cf7e45772ffc4f81952d085e04f05400f6f5fbdbe5ff6: Status 404 returned error can't find the container with id 0834d01ae762db02e31cf7e45772ffc4f81952d085e04f05400f6f5fbdbe5ff6 Jul 09 15:01:15.445153 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.445129 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7"] Jul 09 15:01:15.456687 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.456658 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n"] Jul 09 15:01:15.458248 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:01:15.458221 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7c673c8_b24c_4d4f_9ef7_1e91c41546af.slice/crio-790990a9431661038195c27ded2ad859dfd35038eb1f1d727d256c5a149330de WatchSource:0}: Error finding container 790990a9431661038195c27ded2ad859dfd35038eb1f1d727d256c5a149330de: Status 404 returned error can't find the container with id 790990a9431661038195c27ded2ad859dfd35038eb1f1d727d256c5a149330de Jul 09 15:01:15.851845 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.851809 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:15.852059 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.852036 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" containerID="cri-o://484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38" gracePeriod=30 Jul 09 15:01:15.852687 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:15.852668 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:16.117554 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.117470 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" event={"ID":"a7c673c8-b24c-4d4f-9ef7-1e91c41546af","Type":"ContainerStarted","Data":"790990a9431661038195c27ded2ad859dfd35038eb1f1d727d256c5a149330de"} Jul 09 15:01:16.118817 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.118787 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" event={"ID":"a42114fe-c4a0-4d5a-8e1d-192845f031c8","Type":"ContainerStarted","Data":"0834d01ae762db02e31cf7e45772ffc4f81952d085e04f05400f6f5fbdbe5ff6"} Jul 09 15:01:16.772200 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.772128 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:01:16.772457 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.772208 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:01:16.772457 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.772234 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:01:16.846413 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:16.846391 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:17.014741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014658 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5ns8\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014702 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014746 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014781 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014820 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014860 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.014983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.014906 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert\") pod \"83a8037f-0398-4c2d-ad9f-10e34592b26f\" (UID: \"83a8037f-0398-4c2d-ad9f-10e34592b26f\") " Jul 09 15:01:17.015581 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.015547 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:01:17.018151 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.018124 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:01:17.018259 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.018152 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8" (OuterVolumeSpecName: "kube-api-access-f5ns8") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "kube-api-access-f5ns8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:17.018525 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.018500 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token" (OuterVolumeSpecName: "istio-token") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:17.018619 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.018553 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts" (OuterVolumeSpecName: "cacerts") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:01:17.019181 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.019141 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:01:17.019980 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.019943 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs" (OuterVolumeSpecName: "local-certs") pod "83a8037f-0398-4c2d-ad9f-10e34592b26f" (UID: "83a8037f-0398-4c2d-ad9f-10e34592b26f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:01:17.116348 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116312 2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83a8037f-0398-4c2d-ad9f-10e34592b26f-local-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116348 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116350 2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-cacerts\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116348 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116364 2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-kubeconfig\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116642 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116395 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116642 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116408 2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-ca-configmap\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116642 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116417 2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83a8037f-0398-4c2d-ad9f-10e34592b26f-istio-csr-dns-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.116642 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.116425 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5ns8\" (UniqueName: \"kubernetes.io/projected/83a8037f-0398-4c2d-ad9f-10e34592b26f-kube-api-access-f5ns8\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:01:17.127035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.126833 2558 generic.go:356] "Generic (PLEG): container finished" podID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerID="484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38" exitCode=0 Jul 09 15:01:17.127035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.126897 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" event={"ID":"83a8037f-0398-4c2d-ad9f-10e34592b26f","Type":"ContainerDied","Data":"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38"} Jul 09 15:01:17.127035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.126918 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" Jul 09 15:01:17.127035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.127002 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wz72p" event={"ID":"83a8037f-0398-4c2d-ad9f-10e34592b26f","Type":"ContainerDied","Data":"bc0c58203473bd342a4fe2bce118d1a6039986dd1280d06b37940b8cf02b8e8d"} Jul 09 15:01:17.127035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.127039 2558 scope.go:117] "RemoveContainer" containerID="484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38" Jul 09 15:01:17.129663 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.129633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" event={"ID":"f8e766ed-bc9a-494f-af79-b469aa2ad1a4","Type":"ContainerStarted","Data":"c61a4a850cf2000081d320ad18b55e017324c470448587071c05cf35d824c822"} Jul 09 15:01:17.169081 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.169044 2558 scope.go:117] "RemoveContainer" containerID="484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38" Jul 09 15:01:17.169519 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:01:17.169444 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38\": container with ID starting with 484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38 not found: ID does not exist" containerID="484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38" Jul 09 15:01:17.169519 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.169485 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38"} err="failed to get container status \"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38\": rpc error: code = NotFound desc = could not find container \"484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38\": container with ID starting with 484ad6583cb2a537a0fc571c225320cc5341d1f917935ccc3e10919a74732d38 not found: ID does not exist" Jul 09 15:01:17.213581 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.213520 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" podStartSLOduration=1.6153300320000001 podStartE2EDuration="4.213493807s" totalImagesPullingTime="2.598163775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:17.212132805 +0000 UTC m=+410.075139975" watchObservedRunningTime="2026-07-09 15:01:17.213493807 +0000 UTC m=+410.076500977" Jul 09 15:01:17.260927 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.260893 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:17.278417 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.278348 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wz72p"] Jul 09 15:01:17.624990 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:17.624896 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" path="/var/lib/kubelet/pods/83a8037f-0398-4c2d-ad9f-10e34592b26f/volumes" Jul 09 15:01:18.004102 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:18.004058 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:18.010299 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:18.010275 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:18.134808 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:18.134780 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:18.135971 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:18.135942 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mt8xg" Jul 09 15:01:19.140934 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:19.140859 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" event={"ID":"a42114fe-c4a0-4d5a-8e1d-192845f031c8","Type":"ContainerStarted","Data":"ff914da3cdaedaf24f600e903b650328a92cd71c5d76aa6b17d48168a8b6664b"} Jul 09 15:01:19.141348 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:19.141042 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:19.165654 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:19.165544 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" podStartSLOduration=1.707557698 podStartE2EDuration="5.165522681s" totalImagesPullingTime="3.457964983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:19.165298537 +0000 UTC m=+412.028305708" watchObservedRunningTime="2026-07-09 15:01:19.165522681 +0000 UTC m=+412.028529853" Jul 09 15:01:20.146317 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:20.146278 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" event={"ID":"a7c673c8-b24c-4d4f-9ef7-1e91c41546af","Type":"ContainerStarted","Data":"183941c5c972150857b04633ee00d8ad607ed5ce39d6b9766de1d6782aa070bc"} Jul 09 15:01:20.146728 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:20.146377 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:01:20.170011 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:20.169964 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" podStartSLOduration=2.013697868 podStartE2EDuration="6.169948424s" totalImagesPullingTime="4.156250556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:01:20.167755768 +0000 UTC m=+413.030762972" watchObservedRunningTime="2026-07-09 15:01:20.169948424 +0000 UTC m=+413.032955592" Jul 09 15:01:21.092400 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:21.092358 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lpc4j" Jul 09 15:01:23.102774 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:23.102745 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmn2x" Jul 09 15:01:50.152032 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:50.151957 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-dfzr7" Jul 09 15:01:51.152600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:01:51.152573 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6h24n" Jul 09 15:02:05.114798 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.114762 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:05.115177 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.115071 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" Jul 09 15:02:05.115177 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.115083 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" Jul 09 15:02:05.115177 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.115138 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a8037f-0398-4c2d-ad9f-10e34592b26f" containerName="discovery" Jul 09 15:02:05.118104 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.118088 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:05.121356 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.121326 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 15:02:05.121478 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.121354 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dcp92\"" Jul 09 15:02:05.121478 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.121358 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 15:02:05.130078 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.130056 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:05.164851 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.164823 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82j5p\" (UniqueName: \"kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p\") pod \"kuadrant-operator-catalog-qgffd\" (UID: \"5f175bec-bb75-4974-b356-1f941d600c3d\") " pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:05.265560 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.265528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82j5p\" (UniqueName: \"kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p\") pod \"kuadrant-operator-catalog-qgffd\" (UID: \"5f175bec-bb75-4974-b356-1f941d600c3d\") " pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:05.275071 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.275041 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82j5p\" (UniqueName: \"kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p\") pod \"kuadrant-operator-catalog-qgffd\" (UID: \"5f175bec-bb75-4974-b356-1f941d600c3d\") " pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:05.431345 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.426904 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:05.473344 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.473308 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:05.557905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.557865 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:05.560380 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:05.560355 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f175bec_bb75_4974_b356_1f941d600c3d.slice/crio-44204cee5663039ebccf2d0b12121fa0bb7820a88f63856eb25551563d761860 WatchSource:0}: Error finding container 44204cee5663039ebccf2d0b12121fa0bb7820a88f63856eb25551563d761860: Status 404 returned error can't find the container with id 44204cee5663039ebccf2d0b12121fa0bb7820a88f63856eb25551563d761860 Jul 09 15:02:05.686979 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.686904 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7j9pg"] Jul 09 15:02:05.689795 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.689781 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:05.698247 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.698226 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7j9pg"] Jul 09 15:02:05.770126 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.770098 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldzgz\" (UniqueName: \"kubernetes.io/projected/39e1dc0b-d5aa-469d-8c83-aa3f5aca7133-kube-api-access-ldzgz\") pod \"kuadrant-operator-catalog-7j9pg\" (UID: \"39e1dc0b-d5aa-469d-8c83-aa3f5aca7133\") " pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:05.871326 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.871296 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldzgz\" (UniqueName: \"kubernetes.io/projected/39e1dc0b-d5aa-469d-8c83-aa3f5aca7133-kube-api-access-ldzgz\") pod \"kuadrant-operator-catalog-7j9pg\" (UID: \"39e1dc0b-d5aa-469d-8c83-aa3f5aca7133\") " pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:05.880550 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.880519 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldzgz\" (UniqueName: \"kubernetes.io/projected/39e1dc0b-d5aa-469d-8c83-aa3f5aca7133-kube-api-access-ldzgz\") pod \"kuadrant-operator-catalog-7j9pg\" (UID: \"39e1dc0b-d5aa-469d-8c83-aa3f5aca7133\") " pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:05.999048 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:05.999001 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:06.125097 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:06.125073 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7j9pg"] Jul 09 15:02:06.127062 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:06.127035 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e1dc0b_d5aa_469d_8c83_aa3f5aca7133.slice/crio-e6bbbed70b18905301c5a514d8aca888cad6b647ad90233b6fd959389cb09e49 WatchSource:0}: Error finding container e6bbbed70b18905301c5a514d8aca888cad6b647ad90233b6fd959389cb09e49: Status 404 returned error can't find the container with id e6bbbed70b18905301c5a514d8aca888cad6b647ad90233b6fd959389cb09e49 Jul 09 15:02:06.341591 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:06.341507 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" event={"ID":"5f175bec-bb75-4974-b356-1f941d600c3d","Type":"ContainerStarted","Data":"44204cee5663039ebccf2d0b12121fa0bb7820a88f63856eb25551563d761860"} Jul 09 15:02:06.342730 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:06.342705 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" event={"ID":"39e1dc0b-d5aa-469d-8c83-aa3f5aca7133","Type":"ContainerStarted","Data":"e6bbbed70b18905301c5a514d8aca888cad6b647ad90233b6fd959389cb09e49"} Jul 09 15:02:08.353453 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.353409 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" event={"ID":"5f175bec-bb75-4974-b356-1f941d600c3d","Type":"ContainerStarted","Data":"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063"} Jul 09 15:02:08.353453 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.353435 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" podUID="5f175bec-bb75-4974-b356-1f941d600c3d" containerName="registry-server" containerID="cri-o://21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063" gracePeriod=2 Jul 09 15:02:08.354852 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.354828 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" event={"ID":"39e1dc0b-d5aa-469d-8c83-aa3f5aca7133","Type":"ContainerStarted","Data":"74944bda1216e0fff806e36c9c5bba682dc3604f6994a47d2a09561ff410e791"} Jul 09 15:02:08.372170 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.372128 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" podStartSLOduration=1.09179222 podStartE2EDuration="3.372111413s" totalImagesPullingTime="2.280319193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:08.370897804 +0000 UTC m=+461.233904996" watchObservedRunningTime="2026-07-09 15:02:08.372111413 +0000 UTC m=+461.235118583" Jul 09 15:02:08.388308 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.388270 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" podStartSLOduration=1.6636328219999998 podStartE2EDuration="3.388256362s" totalImagesPullingTime="1.72462354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:08.387971118 +0000 UTC m=+461.250978299" watchObservedRunningTime="2026-07-09 15:02:08.388256362 +0000 UTC m=+461.251263551" Jul 09 15:02:08.595275 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.595252 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:08.696384 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.696295 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82j5p\" (UniqueName: \"kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p\") pod \"5f175bec-bb75-4974-b356-1f941d600c3d\" (UID: \"5f175bec-bb75-4974-b356-1f941d600c3d\") " Jul 09 15:02:08.698370 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.698350 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p" (OuterVolumeSpecName: "kube-api-access-82j5p") pod "5f175bec-bb75-4974-b356-1f941d600c3d" (UID: "5f175bec-bb75-4974-b356-1f941d600c3d"). InnerVolumeSpecName "kube-api-access-82j5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:02:08.796921 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:08.796859 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82j5p\" (UniqueName: \"kubernetes.io/projected/5f175bec-bb75-4974-b356-1f941d600c3d-kube-api-access-82j5p\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:02:09.360384 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.360349 2558 generic.go:356] "Generic (PLEG): container finished" podID="5f175bec-bb75-4974-b356-1f941d600c3d" containerID="21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063" exitCode=0 Jul 09 15:02:09.360844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.360403 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" Jul 09 15:02:09.360844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.360434 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" event={"ID":"5f175bec-bb75-4974-b356-1f941d600c3d","Type":"ContainerDied","Data":"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063"} Jul 09 15:02:09.360844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.360478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qgffd" event={"ID":"5f175bec-bb75-4974-b356-1f941d600c3d","Type":"ContainerDied","Data":"44204cee5663039ebccf2d0b12121fa0bb7820a88f63856eb25551563d761860"} Jul 09 15:02:09.360844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.360497 2558 scope.go:117] "RemoveContainer" containerID="21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063" Jul 09 15:02:09.369678 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.369662 2558 scope.go:117] "RemoveContainer" containerID="21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063" Jul 09 15:02:09.369897 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:09.369864 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063\": container with ID starting with 21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063 not found: ID does not exist" containerID="21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063" Jul 09 15:02:09.369951 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.369904 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063"} err="failed to get container status \"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063\": rpc error: code = NotFound desc = could not find container \"21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063\": container with ID starting with 21d65dc9f2927b549eec224606382746c785a7cea5e6b7dfdd54843d2cca9063 not found: ID does not exist" Jul 09 15:02:09.382757 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.382735 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:09.388826 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.388806 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qgffd"] Jul 09 15:02:09.623706 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:09.623637 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f175bec-bb75-4974-b356-1f941d600c3d" path="/var/lib/kubelet/pods/5f175bec-bb75-4974-b356-1f941d600c3d/volumes" Jul 09 15:02:15.999564 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:15.999520 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:15.999995 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:15.999598 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:16.021532 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:16.021504 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:16.410468 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:16.410398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7j9pg" Jul 09 15:02:38.301666 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.301626 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8"] Jul 09 15:02:38.302191 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.302103 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f175bec-bb75-4974-b356-1f941d600c3d" containerName="registry-server" Jul 09 15:02:38.302191 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.302121 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f175bec-bb75-4974-b356-1f941d600c3d" containerName="registry-server" Jul 09 15:02:38.302191 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.302190 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f175bec-bb75-4974-b356-1f941d600c3d" containerName="registry-server" Jul 09 15:02:38.305957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.305937 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.324649 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.324628 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kg9w2\"" Jul 09 15:02:38.330783 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.330757 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8"] Jul 09 15:02:38.442205 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.442173 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbszc\" (UniqueName: \"kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.442350 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.442241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.543074 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.543043 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.543240 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.543094 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbszc\" (UniqueName: \"kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.543411 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.543391 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.554419 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.554362 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbszc\" (UniqueName: \"kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.615338 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.615314 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:38.775749 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:38.775721 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8"] Jul 09 15:02:38.775888 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:38.775851 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc43e60_d5d9_4114_aa0e_ae25955e0b44.slice/crio-f27bba4affa36a654018a23346e03bd81eb3405d27e69cd5a16f144a1c4c302a WatchSource:0}: Error finding container f27bba4affa36a654018a23346e03bd81eb3405d27e69cd5a16f144a1c4c302a: Status 404 returned error can't find the container with id f27bba4affa36a654018a23346e03bd81eb3405d27e69cd5a16f144a1c4c302a Jul 09 15:02:39.488762 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:39.488727 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" event={"ID":"8dc43e60-d5d9-4114-aa0e-ae25955e0b44","Type":"ContainerStarted","Data":"f27bba4affa36a654018a23346e03bd81eb3405d27e69cd5a16f144a1c4c302a"} Jul 09 15:02:40.759553 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.759522 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6"] Jul 09 15:02:40.762055 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.762024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:40.765201 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.765180 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nxhjd\"" Jul 09 15:02:40.777407 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.777385 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6"] Jul 09 15:02:40.863443 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.863410 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx562\" (UniqueName: \"kubernetes.io/projected/1fe21115-fa3a-4985-8da5-385d942ebc84-kube-api-access-rx562\") pod \"authorino-operator-75cc8d5cb7-ngwg6\" (UID: \"1fe21115-fa3a-4985-8da5-385d942ebc84\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:40.964355 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.964327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx562\" (UniqueName: \"kubernetes.io/projected/1fe21115-fa3a-4985-8da5-385d942ebc84-kube-api-access-rx562\") pod \"authorino-operator-75cc8d5cb7-ngwg6\" (UID: \"1fe21115-fa3a-4985-8da5-385d942ebc84\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:40.981825 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:40.981793 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx562\" (UniqueName: \"kubernetes.io/projected/1fe21115-fa3a-4985-8da5-385d942ebc84-kube-api-access-rx562\") pod \"authorino-operator-75cc8d5cb7-ngwg6\" (UID: \"1fe21115-fa3a-4985-8da5-385d942ebc84\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:41.074251 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:41.074169 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:41.247823 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:41.247792 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6"] Jul 09 15:02:41.249359 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:41.249333 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe21115_fa3a_4985_8da5_385d942ebc84.slice/crio-3af54fecaf13570b4c05466de839042fae127bbf162459c60f2fab12b0001db5 WatchSource:0}: Error finding container 3af54fecaf13570b4c05466de839042fae127bbf162459c60f2fab12b0001db5: Status 404 returned error can't find the container with id 3af54fecaf13570b4c05466de839042fae127bbf162459c60f2fab12b0001db5 Jul 09 15:02:41.500463 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:41.500425 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" event={"ID":"1fe21115-fa3a-4985-8da5-385d942ebc84","Type":"ContainerStarted","Data":"3af54fecaf13570b4c05466de839042fae127bbf162459c60f2fab12b0001db5"} Jul 09 15:02:43.073415 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.073378 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx"] Jul 09 15:02:43.080037 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.080013 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:43.083429 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.083406 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ld9dz\"" Jul 09 15:02:43.083555 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.083436 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 15:02:43.085345 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.085321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zddmg\" (UniqueName: \"kubernetes.io/projected/4b0659de-50e1-4250-b551-0b1b38aeab56-kube-api-access-zddmg\") pod \"dns-operator-controller-manager-864dd78c4c-h59kx\" (UID: \"4b0659de-50e1-4250-b551-0b1b38aeab56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:43.090533 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.090511 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx"] Jul 09 15:02:43.185838 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.185798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zddmg\" (UniqueName: \"kubernetes.io/projected/4b0659de-50e1-4250-b551-0b1b38aeab56-kube-api-access-zddmg\") pod \"dns-operator-controller-manager-864dd78c4c-h59kx\" (UID: \"4b0659de-50e1-4250-b551-0b1b38aeab56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:43.197852 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.197826 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zddmg\" (UniqueName: \"kubernetes.io/projected/4b0659de-50e1-4250-b551-0b1b38aeab56-kube-api-access-zddmg\") pod \"dns-operator-controller-manager-864dd78c4c-h59kx\" (UID: \"4b0659de-50e1-4250-b551-0b1b38aeab56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:43.393657 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.393568 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:43.698824 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:43.698789 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx"] Jul 09 15:02:43.701759 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:43.701734 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0659de_50e1_4250_b551_0b1b38aeab56.slice/crio-c6250a5fb8a2abd34a97e57c71490b091f6f9de96e0fb6a4615ddfe0c2787760 WatchSource:0}: Error finding container c6250a5fb8a2abd34a97e57c71490b091f6f9de96e0fb6a4615ddfe0c2787760: Status 404 returned error can't find the container with id c6250a5fb8a2abd34a97e57c71490b091f6f9de96e0fb6a4615ddfe0c2787760 Jul 09 15:02:44.516831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:44.516746 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" event={"ID":"1fe21115-fa3a-4985-8da5-385d942ebc84","Type":"ContainerStarted","Data":"debc8191a745acab8b97b545887845b05804589acc31bcbd55857439ae801cf3"} Jul 09 15:02:44.517332 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:44.516849 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:44.518291 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:44.518265 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" event={"ID":"4b0659de-50e1-4250-b551-0b1b38aeab56","Type":"ContainerStarted","Data":"c6250a5fb8a2abd34a97e57c71490b091f6f9de96e0fb6a4615ddfe0c2787760"} Jul 09 15:02:44.543726 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:44.543670 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" podStartSLOduration=2.222118499 podStartE2EDuration="4.543650041s" totalImagesPullingTime="2.321531542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:44.540976846 +0000 UTC m=+497.403984013" watchObservedRunningTime="2026-07-09 15:02:44.543650041 +0000 UTC m=+497.406657211" Jul 09 15:02:45.311630 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.311588 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl"] Jul 09 15:02:45.314136 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.314114 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:45.317863 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.317825 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nqb27\"" Jul 09 15:02:45.331527 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.331504 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl"] Jul 09 15:02:45.401024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.400994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfsxd\" (UniqueName: \"kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd\") pod \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" (UID: \"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:45.501846 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.501799 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfsxd\" (UniqueName: \"kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd\") pod \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" (UID: \"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:45.516642 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.516611 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfsxd\" (UniqueName: \"kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd\") pod \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" (UID: \"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:45.628155 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.628125 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:45.961348 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:45.961326 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl"] Jul 09 15:02:45.981734 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:45.981706 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d4ede2_69b9_4561_8b4b_a93ab9bd70ce.slice/crio-8400a72a1fd1482ee800864fb6e13790825dd0879c730941331a4c76834a1c6b WatchSource:0}: Error finding container 8400a72a1fd1482ee800864fb6e13790825dd0879c730941331a4c76834a1c6b: Status 404 returned error can't find the container with id 8400a72a1fd1482ee800864fb6e13790825dd0879c730941331a4c76834a1c6b Jul 09 15:02:46.531206 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.531055 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" event={"ID":"4b0659de-50e1-4250-b551-0b1b38aeab56","Type":"ContainerStarted","Data":"48f89a59eacc4b47ad470f32217db6c59c94372bd65bd9feca8ec02e2c399199"} Jul 09 15:02:46.531366 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.531215 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:46.533193 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.533164 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" event={"ID":"8dc43e60-d5d9-4114-aa0e-ae25955e0b44","Type":"ContainerStarted","Data":"af670155fbe4976223b70bf0817283fd5529ccebcea54af337e9d2dbfa9198cb"} Jul 09 15:02:46.533314 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.533220 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:46.534414 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.534394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" event={"ID":"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce","Type":"ContainerStarted","Data":"8400a72a1fd1482ee800864fb6e13790825dd0879c730941331a4c76834a1c6b"} Jul 09 15:02:46.578946 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.578864 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" podStartSLOduration=0.903269821 podStartE2EDuration="3.578841734s" totalImagesPullingTime="2.675571913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:46.552949515 +0000 UTC m=+499.415956695" watchObservedRunningTime="2026-07-09 15:02:46.578841734 +0000 UTC m=+499.441848903" Jul 09 15:02:46.581204 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:46.581155 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" podStartSLOduration=1.498987702 podStartE2EDuration="8.581136732s" totalImagesPullingTime="7.08214903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:46.578311205 +0000 UTC m=+499.441318375" watchObservedRunningTime="2026-07-09 15:02:46.581136732 +0000 UTC m=+499.444143904" Jul 09 15:02:48.548127 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:48.548075 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" event={"ID":"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce","Type":"ContainerStarted","Data":"2be9670d33d4f98fa0f0a2d83d202f9e8bd8d1a2d9a27384622a1d58bbf72e36"} Jul 09 15:02:48.548697 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:48.548308 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:48.575848 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:48.575777 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" podStartSLOduration=1.803137666 podStartE2EDuration="3.575754867s" totalImagesPullingTime="1.772617201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:48.574043844 +0000 UTC m=+501.437051013" watchObservedRunningTime="2026-07-09 15:02:48.575754867 +0000 UTC m=+501.438762117" Jul 09 15:02:49.489499 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.489455 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5"] Jul 09 15:02:49.492017 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.491997 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.494982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.494946 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 15:02:49.494982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.494960 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 15:02:49.495218 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.494958 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tnv95\"" Jul 09 15:02:49.507082 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.507048 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5"] Jul 09 15:02:49.529435 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.529397 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5bjj\" (UniqueName: \"kubernetes.io/projected/66b4526c-8aa9-4466-884f-6c091b4e9c6a-kube-api-access-p5bjj\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.529670 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.529522 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b4526c-8aa9-4466-884f-6c091b4e9c6a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.529670 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.529556 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.630857 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.630802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b4526c-8aa9-4466-884f-6c091b4e9c6a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.630857 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.630864 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.631533 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.630958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5bjj\" (UniqueName: \"kubernetes.io/projected/66b4526c-8aa9-4466-884f-6c091b4e9c6a-kube-api-access-p5bjj\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.631533 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:49.631019 2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 15:02:49.631533 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:49.631115 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert podName:66b4526c-8aa9-4466-884f-6c091b4e9c6a nodeName:}" failed. No retries permitted until 2026-07-09 15:02:50.131086711 +0000 UTC m=+502.994093861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-qt8b5" (UID: "66b4526c-8aa9-4466-884f-6c091b4e9c6a") : secret "plugin-serving-cert" not found Jul 09 15:02:49.631656 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.631643 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66b4526c-8aa9-4466-884f-6c091b4e9c6a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:49.642044 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:49.642011 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5bjj\" (UniqueName: \"kubernetes.io/projected/66b4526c-8aa9-4466-884f-6c091b4e9c6a-kube-api-access-p5bjj\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:50.134474 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:50.134414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:50.137394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:50.137368 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/66b4526c-8aa9-4466-884f-6c091b4e9c6a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qt8b5\" (UID: \"66b4526c-8aa9-4466-884f-6c091b4e9c6a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:50.402343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:50.402226 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" Jul 09 15:02:50.551219 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:50.551184 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5"] Jul 09 15:02:50.553760 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:02:50.553727 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66b4526c_8aa9_4466_884f_6c091b4e9c6a.slice/crio-403dd1cd25a7075ef262856877969f1ca2d1ad33181dcd2d4144a3b4b1974b8e WatchSource:0}: Error finding container 403dd1cd25a7075ef262856877969f1ca2d1ad33181dcd2d4144a3b4b1974b8e: Status 404 returned error can't find the container with id 403dd1cd25a7075ef262856877969f1ca2d1ad33181dcd2d4144a3b4b1974b8e Jul 09 15:02:50.561628 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:50.561583 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" event={"ID":"66b4526c-8aa9-4466-884f-6c091b4e9c6a","Type":"ContainerStarted","Data":"403dd1cd25a7075ef262856877969f1ca2d1ad33181dcd2d4144a3b4b1974b8e"} Jul 09 15:02:55.526931 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:55.526891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ngwg6" Jul 09 15:02:57.542921 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:57.542888 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h59kx" Jul 09 15:02:57.543322 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:57.542976 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.372323 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.372281 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8"] Jul 09 15:02:59.372921 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.372539 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" containerName="manager" containerID="cri-o://af670155fbe4976223b70bf0817283fd5529ccebcea54af337e9d2dbfa9198cb" gracePeriod=2 Jul 09 15:02:59.384112 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.384053 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8"] Jul 09 15:02:59.399067 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.399024 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl"] Jul 09 15:02:59.399299 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.399258 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" containerName="manager" containerID="cri-o://2be9670d33d4f98fa0f0a2d83d202f9e8bd8d1a2d9a27384622a1d58bbf72e36" gracePeriod=2 Jul 09 15:02:59.401628 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.401607 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:59.402717 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.402637 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.410361 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.410317 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:02:59.411416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.411091 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" containerName="manager" Jul 09 15:02:59.411416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.411114 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" containerName="manager" Jul 09 15:02:59.411416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.411211 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" containerName="manager" Jul 09 15:02:59.414539 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.414522 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl"] Jul 09 15:02:59.414636 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.414625 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.418708 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.418679 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:59.422284 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.422246 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.426363 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.426343 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:02:59.426448 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.426372 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.430137 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.430104 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:59.430237 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.430220 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd"] Jul 09 15:02:59.430604 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.430576 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" containerName="manager" Jul 09 15:02:59.430604 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.430595 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" containerName="manager" Jul 09 15:02:59.430765 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.430671 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" containerName="manager" Jul 09 15:02:59.433467 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.433450 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:02:59.455369 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.455337 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd"] Jul 09 15:02:59.471021 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.470985 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5htc8\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.476579 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:02:59.476548 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:59.505174 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.505022 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl2gc\" (UniqueName: \"kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.505174 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.505066 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.505174 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.505092 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhb5r\" (UniqueName: \"kubernetes.io/projected/c9040363-d694-4579-8a43-306e99815a98-kube-api-access-dhb5r\") pod \"limitador-operator-controller-manager-859d7b78bd-6vvcd\" (UID: \"c9040363-d694-4579-8a43-306e99815a98\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:02:59.605775 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.605740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl2gc\" (UniqueName: \"kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.605957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.605788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.605957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.605903 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhb5r\" (UniqueName: \"kubernetes.io/projected/c9040363-d694-4579-8a43-306e99815a98-kube-api-access-dhb5r\") pod \"limitador-operator-controller-manager-859d7b78bd-6vvcd\" (UID: \"c9040363-d694-4579-8a43-306e99815a98\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:02:59.606195 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.606162 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.608714 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.608643 2558 generic.go:356] "Generic (PLEG): container finished" podID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" containerID="af670155fbe4976223b70bf0817283fd5529ccebcea54af337e9d2dbfa9198cb" exitCode=0 Jul 09 15:02:59.610427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.610406 2558 generic.go:356] "Generic (PLEG): container finished" podID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" containerID="2be9670d33d4f98fa0f0a2d83d202f9e8bd8d1a2d9a27384622a1d58bbf72e36" exitCode=0 Jul 09 15:02:59.624417 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.624357 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhb5r\" (UniqueName: \"kubernetes.io/projected/c9040363-d694-4579-8a43-306e99815a98-kube-api-access-dhb5r\") pod \"limitador-operator-controller-manager-859d7b78bd-6vvcd\" (UID: \"c9040363-d694-4579-8a43-306e99815a98\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:02:59.627492 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.627466 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl2gc\" (UniqueName: \"kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8q5vj\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.660682 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.660651 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:02:59.663409 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.663390 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:02:59.706864 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.706825 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbszc\" (UniqueName: \"kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc\") pod \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " Jul 09 15:02:59.707034 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.706910 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume\") pod \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\" (UID: \"8dc43e60-d5d9-4114-aa0e-ae25955e0b44\") " Jul 09 15:02:59.707034 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.706950 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfsxd\" (UniqueName: \"kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd\") pod \"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce\" (UID: \"d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce\") " Jul 09 15:02:59.707556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.707501 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8dc43e60-d5d9-4114-aa0e-ae25955e0b44" (UID: "8dc43e60-d5d9-4114-aa0e-ae25955e0b44"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:02:59.709298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.709271 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd" (OuterVolumeSpecName: "kube-api-access-hfsxd") pod "d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" (UID: "d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce"). InnerVolumeSpecName "kube-api-access-hfsxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:02:59.709398 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.709305 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc" (OuterVolumeSpecName: "kube-api-access-sbszc") pod "8dc43e60-d5d9-4114-aa0e-ae25955e0b44" (UID: "8dc43e60-d5d9-4114-aa0e-ae25955e0b44"). InnerVolumeSpecName "kube-api-access-sbszc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:02:59.793818 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.793776 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:02:59.802422 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.802395 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:02:59.807600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.807581 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfsxd\" (UniqueName: \"kubernetes.io/projected/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce-kube-api-access-hfsxd\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:02:59.807698 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.807602 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbszc\" (UniqueName: \"kubernetes.io/projected/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-kube-api-access-sbszc\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:02:59.807698 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:02:59.807613 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dc43e60-d5d9-4114-aa0e-ae25955e0b44-extensions-socket-volume\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:03:00.469675 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.469645 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:03:00.476551 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.476529 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.489438 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.489412 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:03:00.512030 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.511953 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.512030 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.512000 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgwj\" (UniqueName: \"kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.613103 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.613064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.613278 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.613116 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgwj\" (UniqueName: \"kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.613556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.613513 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.616662 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.616637 2558 scope.go:117] "RemoveContainer" containerID="af670155fbe4976223b70bf0817283fd5529ccebcea54af337e9d2dbfa9198cb" Jul 09 15:03:00.616788 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.616637 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5htc8" Jul 09 15:03:00.618397 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.618377 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:03:00.624214 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.624189 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgwj\" (UniqueName: \"kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj\") pod \"kuadrant-operator-controller-manager-86878c769f-l89sh\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:00.642091 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:03:00.642031 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:03:00.645089 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:03:00.645062 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fpjhl\" is forbidden: User \"system:node:ip-10-0-142-231.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-231.ec2.internal' and this object" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fpjhl" Jul 09 15:03:00.789068 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:00.789023 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:01.623640 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:01.623605 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dc43e60-d5d9-4114-aa0e-ae25955e0b44" path="/var/lib/kubelet/pods/8dc43e60-d5d9-4114-aa0e-ae25955e0b44/volumes" Jul 09 15:03:01.624063 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:01.624046 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce" path="/var/lib/kubelet/pods/d7d4ede2-69b9-4561-8b4b-a93ab9bd70ce/volumes" Jul 09 15:03:14.079777 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.079620 2558 scope.go:117] "RemoveContainer" containerID="2be9670d33d4f98fa0f0a2d83d202f9e8bd8d1a2d9a27384622a1d58bbf72e36" Jul 09 15:03:14.275196 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.275084 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:03:14.276691 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:03:14.276660 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fb8602_f3c8_4948_9e8c_05a3af84c634.slice/crio-46e5e277adb451a8419ba736365ea729f9be7ceb311e8b782c576de4a05e46d8 WatchSource:0}: Error finding container 46e5e277adb451a8419ba736365ea729f9be7ceb311e8b782c576de4a05e46d8: Status 404 returned error can't find the container with id 46e5e277adb451a8419ba736365ea729f9be7ceb311e8b782c576de4a05e46d8 Jul 09 15:03:14.302238 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.295644 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:03:14.323891 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.323844 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd"] Jul 09 15:03:14.327114 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:03:14.327068 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9040363_d694_4579_8a43_306e99815a98.slice/crio-2b627461468c2273a64dd02a672f0d2ce98427a8da4853688cc640b38f0d1cea WatchSource:0}: Error finding container 2b627461468c2273a64dd02a672f0d2ce98427a8da4853688cc640b38f0d1cea: Status 404 returned error can't find the container with id 2b627461468c2273a64dd02a672f0d2ce98427a8da4853688cc640b38f0d1cea Jul 09 15:03:14.689320 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.689198 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" event={"ID":"60fb8602-f3c8-4948-9e8c-05a3af84c634","Type":"ContainerStarted","Data":"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848"} Jul 09 15:03:14.689320 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.689276 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" event={"ID":"60fb8602-f3c8-4948-9e8c-05a3af84c634","Type":"ContainerStarted","Data":"46e5e277adb451a8419ba736365ea729f9be7ceb311e8b782c576de4a05e46d8"} Jul 09 15:03:14.689616 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.689458 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:03:14.693347 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.693313 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" event={"ID":"c9040363-d694-4579-8a43-306e99815a98","Type":"ContainerStarted","Data":"960c1ed2e5e06ec2b0656fa63bb7d8ae005f894a6b25aa04d24b577d7accaaa1"} Jul 09 15:03:14.693526 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.693371 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" event={"ID":"c9040363-d694-4579-8a43-306e99815a98","Type":"ContainerStarted","Data":"2b627461468c2273a64dd02a672f0d2ce98427a8da4853688cc640b38f0d1cea"} Jul 09 15:03:14.693526 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.693421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:03:14.695046 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.695022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" event={"ID":"66b4526c-8aa9-4466-884f-6c091b4e9c6a","Type":"ContainerStarted","Data":"3c3c3301af1aad0fb6b53c24c30cc9654605f0eae8a2ba1591d7596a4ef16487"} Jul 09 15:03:14.699986 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.699954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" event={"ID":"2937b7a9-127f-4094-9979-a44093ee32e9","Type":"ContainerStarted","Data":"f82db3ede2fe265dda8d68ba5cfe23bbf71432cb489b0398759595556bb81a64"} Jul 09 15:03:14.700135 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.699998 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" event={"ID":"2937b7a9-127f-4094-9979-a44093ee32e9","Type":"ContainerStarted","Data":"f93b97e4d33395108122b6c2c1da5a21a7a9c6f4b598fecb11613b5144c69f30"} Jul 09 15:03:14.700135 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.700119 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:14.716950 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.716871 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" podStartSLOduration=15.71685371 podStartE2EDuration="15.71685371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.714159656 +0000 UTC m=+527.577166825" watchObservedRunningTime="2026-07-09 15:03:14.71685371 +0000 UTC m=+527.579860879" Jul 09 15:03:14.786701 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.786633 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qt8b5" podStartSLOduration=2.11769722 podStartE2EDuration="25.78661197s" totalImagesPullingTime="23.66891475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.786578363 +0000 UTC m=+527.649585532" watchObservedRunningTime="2026-07-09 15:03:14.78661197 +0000 UTC m=+527.649619139" Jul 09 15:03:14.787093 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.787068 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" podStartSLOduration=15.787060765 podStartE2EDuration="15.787060765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.755403902 +0000 UTC m=+527.618411090" watchObservedRunningTime="2026-07-09 15:03:14.787060765 +0000 UTC m=+527.650067936" Jul 09 15:03:14.831351 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:14.831287 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" podStartSLOduration=14.83126681 podStartE2EDuration="14.83126681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.830628479 +0000 UTC m=+527.693635649" watchObservedRunningTime="2026-07-09 15:03:14.83126681 +0000 UTC m=+527.694273978" Jul 09 15:03:25.706743 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:25.706672 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6vvcd" Jul 09 15:03:25.707274 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:25.706755 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:03:25.707274 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:25.706801 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:03:25.832294 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:25.832264 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:03:25.832457 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:25.832431 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" podUID="60fb8602-f3c8-4948-9e8c-05a3af84c634" containerName="manager" containerID="cri-o://e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848" gracePeriod=10 Jul 09 15:03:26.087713 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.087692 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:03:26.243605 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.243579 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume\") pod \"60fb8602-f3c8-4948-9e8c-05a3af84c634\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " Jul 09 15:03:26.243749 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.243684 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl2gc\" (UniqueName: \"kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc\") pod \"60fb8602-f3c8-4948-9e8c-05a3af84c634\" (UID: \"60fb8602-f3c8-4948-9e8c-05a3af84c634\") " Jul 09 15:03:26.244006 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.243984 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "60fb8602-f3c8-4948-9e8c-05a3af84c634" (UID: "60fb8602-f3c8-4948-9e8c-05a3af84c634"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:03:26.245460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.245436 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc" (OuterVolumeSpecName: "kube-api-access-zl2gc") pod "60fb8602-f3c8-4948-9e8c-05a3af84c634" (UID: "60fb8602-f3c8-4948-9e8c-05a3af84c634"). InnerVolumeSpecName "kube-api-access-zl2gc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:03:26.345198 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.345170 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60fb8602-f3c8-4948-9e8c-05a3af84c634-extensions-socket-volume\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:03:26.345198 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.345193 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl2gc\" (UniqueName: \"kubernetes.io/projected/60fb8602-f3c8-4948-9e8c-05a3af84c634-kube-api-access-zl2gc\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:03:26.753641 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.753604 2558 generic.go:356] "Generic (PLEG): container finished" podID="60fb8602-f3c8-4948-9e8c-05a3af84c634" containerID="e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848" exitCode=0 Jul 09 15:03:26.754073 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.753644 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" Jul 09 15:03:26.754073 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.753688 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" event={"ID":"60fb8602-f3c8-4948-9e8c-05a3af84c634","Type":"ContainerDied","Data":"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848"} Jul 09 15:03:26.754073 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.753741 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj" event={"ID":"60fb8602-f3c8-4948-9e8c-05a3af84c634","Type":"ContainerDied","Data":"46e5e277adb451a8419ba736365ea729f9be7ceb311e8b782c576de4a05e46d8"} Jul 09 15:03:26.754073 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.753764 2558 scope.go:117] "RemoveContainer" containerID="e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848" Jul 09 15:03:26.762917 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.762899 2558 scope.go:117] "RemoveContainer" containerID="e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848" Jul 09 15:03:26.763195 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:03:26.763177 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848\": container with ID starting with e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848 not found: ID does not exist" containerID="e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848" Jul 09 15:03:26.763244 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.763203 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848"} err="failed to get container status \"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848\": rpc error: code = NotFound desc = could not find container \"e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848\": container with ID starting with e5d8eaa3184a3d6c26a6efe0ec27eb7c9e0f05335258c723cf65f0b72a86d848 not found: ID does not exist" Jul 09 15:03:26.776272 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.776250 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:03:26.785186 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:26.785166 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8q5vj"] Jul 09 15:03:27.623145 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:27.623118 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60fb8602-f3c8-4948-9e8c-05a3af84c634" path="/var/lib/kubelet/pods/60fb8602-f3c8-4948-9e8c-05a3af84c634/volumes" Jul 09 15:03:47.466918 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.466885 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:47.467394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.467201 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60fb8602-f3c8-4948-9e8c-05a3af84c634" containerName="manager" Jul 09 15:03:47.467394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.467213 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="60fb8602-f3c8-4948-9e8c-05a3af84c634" containerName="manager" Jul 09 15:03:47.467394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.467275 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="60fb8602-f3c8-4948-9e8c-05a3af84c634" containerName="manager" Jul 09 15:03:47.526594 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.526567 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:47.526746 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.526650 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:47.530018 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.529997 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pq76b\"" Jul 09 15:03:47.621347 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.621313 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9g5f\" (UniqueName: \"kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f\") pod \"authorino-58cd955675-9shpr\" (UID: \"ef7f9426-f46a-43c8-9d1f-a42a10a62467\") " pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:47.722715 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.722621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9g5f\" (UniqueName: \"kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f\") pod \"authorino-58cd955675-9shpr\" (UID: \"ef7f9426-f46a-43c8-9d1f-a42a10a62467\") " pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:47.733208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.733183 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9g5f\" (UniqueName: \"kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f\") pod \"authorino-58cd955675-9shpr\" (UID: \"ef7f9426-f46a-43c8-9d1f-a42a10a62467\") " pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:47.834991 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.834964 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:47.967287 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:47.967262 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:47.969072 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:03:47.969045 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef7f9426_f46a_43c8_9d1f_a42a10a62467.slice/crio-5707d348450a150af76cb3681518bd3cac8bfdd2131e7612a583db0ef125a7e6 WatchSource:0}: Error finding container 5707d348450a150af76cb3681518bd3cac8bfdd2131e7612a583db0ef125a7e6: Status 404 returned error can't find the container with id 5707d348450a150af76cb3681518bd3cac8bfdd2131e7612a583db0ef125a7e6 Jul 09 15:03:48.854705 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:48.854664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9shpr" event={"ID":"ef7f9426-f46a-43c8-9d1f-a42a10a62467","Type":"ContainerStarted","Data":"5707d348450a150af76cb3681518bd3cac8bfdd2131e7612a583db0ef125a7e6"} Jul 09 15:03:51.872964 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:51.872863 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9shpr" event={"ID":"ef7f9426-f46a-43c8-9d1f-a42a10a62467","Type":"ContainerStarted","Data":"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6"} Jul 09 15:03:51.891571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:51.891527 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-9shpr" podStartSLOduration=1.398168672 podStartE2EDuration="4.891511033s" totalImagesPullingTime="3.493342361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:51.889963781 +0000 UTC m=+564.752970950" watchObservedRunningTime="2026-07-09 15:03:51.891511033 +0000 UTC m=+564.754518201" Jul 09 15:03:53.631064 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:53.631034 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:53.881536 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:53.881454 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-9shpr" podUID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" containerName="authorino" containerID="cri-o://da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6" gracePeriod=30 Jul 09 15:03:54.127553 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.127528 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:54.173087 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.173014 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9g5f\" (UniqueName: \"kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f\") pod \"ef7f9426-f46a-43c8-9d1f-a42a10a62467\" (UID: \"ef7f9426-f46a-43c8-9d1f-a42a10a62467\") " Jul 09 15:03:54.175125 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.175101 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f" (OuterVolumeSpecName: "kube-api-access-l9g5f") pod "ef7f9426-f46a-43c8-9d1f-a42a10a62467" (UID: "ef7f9426-f46a-43c8-9d1f-a42a10a62467"). InnerVolumeSpecName "kube-api-access-l9g5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:03:54.273541 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.273508 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9g5f\" (UniqueName: \"kubernetes.io/projected/ef7f9426-f46a-43c8-9d1f-a42a10a62467-kube-api-access-l9g5f\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:03:54.886977 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.886943 2558 generic.go:356] "Generic (PLEG): container finished" podID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" containerID="da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6" exitCode=0 Jul 09 15:03:54.887430 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.886987 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9shpr" Jul 09 15:03:54.887430 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.887024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9shpr" event={"ID":"ef7f9426-f46a-43c8-9d1f-a42a10a62467","Type":"ContainerDied","Data":"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6"} Jul 09 15:03:54.887430 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.887067 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9shpr" event={"ID":"ef7f9426-f46a-43c8-9d1f-a42a10a62467","Type":"ContainerDied","Data":"5707d348450a150af76cb3681518bd3cac8bfdd2131e7612a583db0ef125a7e6"} Jul 09 15:03:54.887430 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.887085 2558 scope.go:117] "RemoveContainer" containerID="da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6" Jul 09 15:03:54.896134 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.896117 2558 scope.go:117] "RemoveContainer" containerID="da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6" Jul 09 15:03:54.896402 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:03:54.896384 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6\": container with ID starting with da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6 not found: ID does not exist" containerID="da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6" Jul 09 15:03:54.896445 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.896410 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6"} err="failed to get container status \"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6\": rpc error: code = NotFound desc = could not find container \"da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6\": container with ID starting with da3db11af2e5c9bcdbc33700e0ac85c5d063b11ac3de0871cdf7b697011d2be6 not found: ID does not exist" Jul 09 15:03:54.910595 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.910568 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:54.918468 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:54.918449 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-9shpr"] Jul 09 15:03:55.624169 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:03:55.624130 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" path="/var/lib/kubelet/pods/ef7f9426-f46a-43c8-9d1f-a42a10a62467/volumes" Jul 09 15:04:21.320441 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.320404 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gbjcl"] Jul 09 15:04:21.320819 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.320728 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" containerName="authorino" Jul 09 15:04:21.320819 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.320741 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" containerName="authorino" Jul 09 15:04:21.320819 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.320813 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef7f9426-f46a-43c8-9d1f-a42a10a62467" containerName="authorino" Jul 09 15:04:21.363933 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.363902 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gbjcl"] Jul 09 15:04:21.364083 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.364017 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:21.366988 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.366968 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pq76b\"" Jul 09 15:04:21.498217 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.498184 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2wk\" (UniqueName: \"kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk\") pod \"authorino-5d4b67c446-gbjcl\" (UID: \"2d2ac0ff-c0a0-4e22-aec0-6a76286b9304\") " pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:21.565740 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.565704 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gbjcl"] Jul 09 15:04:21.565989 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:04:21.565962 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jv2wk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-gbjcl" podUID="2d2ac0ff-c0a0-4e22-aec0-6a76286b9304" Jul 09 15:04:21.599540 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.599468 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2wk\" (UniqueName: \"kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk\") pod \"authorino-5d4b67c446-gbjcl\" (UID: \"2d2ac0ff-c0a0-4e22-aec0-6a76286b9304\") " pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:21.604539 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.604517 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56f877cdc8-9rlk2"] Jul 09 15:04:21.619127 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.619100 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:21.620584 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.620558 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2wk\" (UniqueName: \"kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk\") pod \"authorino-5d4b67c446-gbjcl\" (UID: \"2d2ac0ff-c0a0-4e22-aec0-6a76286b9304\") " pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:21.629695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.627413 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56f877cdc8-9rlk2"] Jul 09 15:04:21.687505 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.687476 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56f877cdc8-9rlk2"] Jul 09 15:04:21.687715 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:04:21.687696 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nz6k6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-56f877cdc8-9rlk2" podUID="99092100-b27f-4068-8d6b-3de393e9cbc9" Jul 09 15:04:21.700080 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.700053 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz6k6\" (UniqueName: \"kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6\") pod \"authorino-56f877cdc8-9rlk2\" (UID: \"99092100-b27f-4068-8d6b-3de393e9cbc9\") " pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:21.722233 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.722207 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:04:21.737475 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.737449 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:04:21.737573 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.737539 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.740595 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.740577 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 15:04:21.801318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.801288 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.801476 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.801335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz6k6\" (UniqueName: \"kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6\") pod \"authorino-56f877cdc8-9rlk2\" (UID: \"99092100-b27f-4068-8d6b-3de393e9cbc9\") " pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:21.801476 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.801358 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xkd4\" (UniqueName: \"kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.810746 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.810720 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz6k6\" (UniqueName: \"kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6\") pod \"authorino-56f877cdc8-9rlk2\" (UID: \"99092100-b27f-4068-8d6b-3de393e9cbc9\") " pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:21.901939 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.901848 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xkd4\" (UniqueName: \"kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.902077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.901958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.904336 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.904308 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:21.910718 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:21.910698 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xkd4\" (UniqueName: \"kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4\") pod \"authorino-6df9f875c8-5cftz\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:22.006376 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.006349 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:22.006525 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.006353 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:22.012342 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.012319 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:22.015096 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.015080 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:22.049456 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.049434 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:04:22.103130 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.103103 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz6k6\" (UniqueName: \"kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6\") pod \"99092100-b27f-4068-8d6b-3de393e9cbc9\" (UID: \"99092100-b27f-4068-8d6b-3de393e9cbc9\") " Jul 09 15:04:22.103282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.103152 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv2wk\" (UniqueName: \"kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk\") pod \"2d2ac0ff-c0a0-4e22-aec0-6a76286b9304\" (UID: \"2d2ac0ff-c0a0-4e22-aec0-6a76286b9304\") " Jul 09 15:04:22.105196 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.105161 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6" (OuterVolumeSpecName: "kube-api-access-nz6k6") pod "99092100-b27f-4068-8d6b-3de393e9cbc9" (UID: "99092100-b27f-4068-8d6b-3de393e9cbc9"). InnerVolumeSpecName "kube-api-access-nz6k6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:04:22.105296 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.105201 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk" (OuterVolumeSpecName: "kube-api-access-jv2wk") pod "2d2ac0ff-c0a0-4e22-aec0-6a76286b9304" (UID: "2d2ac0ff-c0a0-4e22-aec0-6a76286b9304"). InnerVolumeSpecName "kube-api-access-jv2wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:04:22.184524 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.184498 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:04:22.186621 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:04:22.186596 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eb73c66_e43d_4541_925d_b52dbd3f6501.slice/crio-7e141258363f085bdd6bc31a2d78a28f35308cebdff28f909facbe90b7362c05 WatchSource:0}: Error finding container 7e141258363f085bdd6bc31a2d78a28f35308cebdff28f909facbe90b7362c05: Status 404 returned error can't find the container with id 7e141258363f085bdd6bc31a2d78a28f35308cebdff28f909facbe90b7362c05 Jul 09 15:04:22.204188 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.204168 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv2wk\" (UniqueName: \"kubernetes.io/projected/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304-kube-api-access-jv2wk\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:04:22.204281 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:22.204190 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz6k6\" (UniqueName: \"kubernetes.io/projected/99092100-b27f-4068-8d6b-3de393e9cbc9-kube-api-access-nz6k6\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:04:23.011929 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.011889 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df9f875c8-5cftz" event={"ID":"1eb73c66-e43d-4541-925d-b52dbd3f6501","Type":"ContainerStarted","Data":"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e"} Jul 09 15:04:23.011929 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.011935 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df9f875c8-5cftz" event={"ID":"1eb73c66-e43d-4541-925d-b52dbd3f6501","Type":"ContainerStarted","Data":"7e141258363f085bdd6bc31a2d78a28f35308cebdff28f909facbe90b7362c05"} Jul 09 15:04:23.012418 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.011908 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f877cdc8-9rlk2" Jul 09 15:04:23.012418 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.012040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gbjcl" Jul 09 15:04:23.033646 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.033604 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6df9f875c8-5cftz" podStartSLOduration=1.6607060900000001 podStartE2EDuration="2.033588799s" totalImagesPullingTime="372.882709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:04:23.032149994 +0000 UTC m=+595.895157163" watchObservedRunningTime="2026-07-09 15:04:23.033588799 +0000 UTC m=+595.896596031" Jul 09 15:04:23.057783 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.057749 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gbjcl"] Jul 09 15:04:23.064171 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.064145 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gbjcl"] Jul 09 15:04:23.091382 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.091356 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56f877cdc8-9rlk2"] Jul 09 15:04:23.095088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.095067 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56f877cdc8-9rlk2"] Jul 09 15:04:23.623412 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.623382 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d2ac0ff-c0a0-4e22-aec0-6a76286b9304" path="/var/lib/kubelet/pods/2d2ac0ff-c0a0-4e22-aec0-6a76286b9304/volumes" Jul 09 15:04:23.623591 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:23.623579 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99092100-b27f-4068-8d6b-3de393e9cbc9" path="/var/lib/kubelet/pods/99092100-b27f-4068-8d6b-3de393e9cbc9/volumes" Jul 09 15:04:27.585132 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:27.585106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:04:27.586827 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:04:27.586808 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:05:03.218993 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.218914 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qdsb8"] Jul 09 15:05:03.222289 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.222272 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.227979 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.227957 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wvsb8\"" Jul 09 15:05:03.229373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.229355 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 15:05:03.248155 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.248128 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qdsb8"] Jul 09 15:05:03.348941 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.348914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8mp6\" (UniqueName: \"kubernetes.io/projected/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-kube-api-access-w8mp6\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.349099 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.348977 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.449398 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.449368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.449550 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.449425 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mp6\" (UniqueName: \"kubernetes.io/projected/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-kube-api-access-w8mp6\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.450180 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.450150 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.461733 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.461711 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mp6\" (UniqueName: \"kubernetes.io/projected/d5768f7e-1987-40ce-a9cc-7f63bcdf1dde-kube-api-access-w8mp6\") pod \"payload-pre-processing-7964457696-qdsb8\" (UID: \"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde\") " pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.532644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.532584 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:03.667886 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:03.667854 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qdsb8"] Jul 09 15:05:03.669612 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:05:03.669589 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5768f7e_1987_40ce_a9cc_7f63bcdf1dde.slice/crio-82e1a7e1dcf4a90788d1f7bf29d3078079eb945c2e511ea2abdd35de68a77dac WatchSource:0}: Error finding container 82e1a7e1dcf4a90788d1f7bf29d3078079eb945c2e511ea2abdd35de68a77dac: Status 404 returned error can't find the container with id 82e1a7e1dcf4a90788d1f7bf29d3078079eb945c2e511ea2abdd35de68a77dac Jul 09 15:05:04.203046 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:04.203012 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" event={"ID":"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde","Type":"ContainerStarted","Data":"82e1a7e1dcf4a90788d1f7bf29d3078079eb945c2e511ea2abdd35de68a77dac"} Jul 09 15:05:06.215854 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:06.215816 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" event={"ID":"d5768f7e-1987-40ce-a9cc-7f63bcdf1dde","Type":"ContainerStarted","Data":"fa47de5821de7ab0f56b27e58dc75faa82bfc5f365143a96cbe86a3386c7bd61"} Jul 09 15:05:06.216249 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:06.215947 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:05:06.234468 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:06.234421 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" podStartSLOduration=1.587258528 podStartE2EDuration="3.234406557s" totalImagesPullingTime="1.647148029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:05:06.233150086 +0000 UTC m=+639.096157257" watchObservedRunningTime="2026-07-09 15:05:06.234406557 +0000 UTC m=+639.097413727" Jul 09 15:05:17.222332 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:05:17.222294 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-qdsb8" Jul 09 15:06:00.405636 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.405606 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs"] Jul 09 15:06:00.409060 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.409038 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.412155 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.412131 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qjgf2\"" Jul 09 15:06:00.412282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.412156 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 15:06:00.413521 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.413503 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 15:06:00.413646 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.413620 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 15:06:00.421467 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.421444 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs"] Jul 09 15:06:00.522500 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.522658 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.522658 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522575 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.522658 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjmt9\" (UniqueName: \"kubernetes.io/projected/4e78d009-16d2-4c08-917e-9d99b0e16066-kube-api-access-mjmt9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.522658 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522637 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.522792 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.522670 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78d009-16d2-4c08-917e-9d99b0e16066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623136 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623107 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78d009-16d2-4c08-917e-9d99b0e16066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623324 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623150 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623324 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623188 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623324 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623324 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623251 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjmt9\" (UniqueName: \"kubernetes.io/projected/4e78d009-16d2-4c08-917e-9d99b0e16066-kube-api-access-mjmt9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623324 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623597 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623563 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623652 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623603 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.623652 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.623644 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.625528 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.625498 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78d009-16d2-4c08-917e-9d99b0e16066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.625671 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.625655 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78d009-16d2-4c08-917e-9d99b0e16066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.633622 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.633602 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjmt9\" (UniqueName: \"kubernetes.io/projected/4e78d009-16d2-4c08-917e-9d99b0e16066-kube-api-access-mjmt9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xmtxs\" (UID: \"4e78d009-16d2-4c08-917e-9d99b0e16066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.720154 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.720125 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:00.852905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.852867 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs"] Jul 09 15:06:00.855432 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:06:00.855405 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e78d009_16d2_4c08_917e_9d99b0e16066.slice/crio-c26c03d04546b5dc246a77f180525673616c4495eb9a49305297e895565f9ba2 WatchSource:0}: Error finding container c26c03d04546b5dc246a77f180525673616c4495eb9a49305297e895565f9ba2: Status 404 returned error can't find the container with id c26c03d04546b5dc246a77f180525673616c4495eb9a49305297e895565f9ba2 Jul 09 15:06:00.856897 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:00.856858 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:06:01.471330 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:01.471291 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" event={"ID":"4e78d009-16d2-4c08-917e-9d99b0e16066","Type":"ContainerStarted","Data":"c26c03d04546b5dc246a77f180525673616c4495eb9a49305297e895565f9ba2"} Jul 09 15:06:02.478303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:02.478264 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" event={"ID":"4e78d009-16d2-4c08-917e-9d99b0e16066","Type":"ContainerStarted","Data":"70723d726e4161a2d1ed0b70d7df4e47ec8679111064a5eb73e4787e38495f05"} Jul 09 15:06:04.185572 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.185511 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" podStartSLOduration=3.058701602 podStartE2EDuration="4.185491248s" totalImagesPullingTime="1.126789646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:06:02.498944383 +0000 UTC m=+695.361951554" watchObservedRunningTime="2026-07-09 15:06:04.185491248 +0000 UTC m=+697.048498417" Jul 09 15:06:04.191916 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.188491 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7"] Jul 09 15:06:04.193484 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.193462 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.196746 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.196730 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 15:06:04.200505 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.200221 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7"] Jul 09 15:06:04.254788 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254753 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.254788 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.255001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254841 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.255001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254907 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgl7\" (UniqueName: \"kubernetes.io/projected/2a0badc3-ba07-41cc-9b74-571d75bec163-kube-api-access-jqgl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.255001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a0badc3-ba07-41cc-9b74-571d75bec163-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.255001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.254968 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355758 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355758 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355719 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgl7\" (UniqueName: \"kubernetes.io/projected/2a0badc3-ba07-41cc-9b74-571d75bec163-kube-api-access-jqgl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355761 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a0badc3-ba07-41cc-9b74-571d75bec163-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355783 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.355989 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.355957 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.356054 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.356032 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.356141 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.356120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.357835 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.357809 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a0badc3-ba07-41cc-9b74-571d75bec163-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.358156 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.358138 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a0badc3-ba07-41cc-9b74-571d75bec163-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.365705 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.365682 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgl7\" (UniqueName: \"kubernetes.io/projected/2a0badc3-ba07-41cc-9b74-571d75bec163-kube-api-access-jqgl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7\" (UID: \"2a0badc3-ba07-41cc-9b74-571d75bec163\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.503780 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.503746 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:04.641264 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:04.641235 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7"] Jul 09 15:06:04.643672 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:06:04.643639 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0badc3_ba07_41cc_9b74_571d75bec163.slice/crio-5d072006a10f851e20d53d464a1ae8d914a7ad45de04b82ec1e1ee110f3a2f1e WatchSource:0}: Error finding container 5d072006a10f851e20d53d464a1ae8d914a7ad45de04b82ec1e1ee110f3a2f1e: Status 404 returned error can't find the container with id 5d072006a10f851e20d53d464a1ae8d914a7ad45de04b82ec1e1ee110f3a2f1e Jul 09 15:06:05.492927 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:05.492889 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" event={"ID":"2a0badc3-ba07-41cc-9b74-571d75bec163","Type":"ContainerStarted","Data":"5746fffb79a7d4d09732c59d1ad1709cb2905b91ff8e6dd92d91bd21891de5fc"} Jul 09 15:06:05.493308 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:05.492946 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" event={"ID":"2a0badc3-ba07-41cc-9b74-571d75bec163","Type":"ContainerStarted","Data":"5d072006a10f851e20d53d464a1ae8d914a7ad45de04b82ec1e1ee110f3a2f1e"} Jul 09 15:06:05.513622 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:05.513569 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" podStartSLOduration=1.252016408 podStartE2EDuration="1.513551137s" totalImagesPullingTime="261.534729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:06:05.512607776 +0000 UTC m=+698.375614970" watchObservedRunningTime="2026-07-09 15:06:05.513551137 +0000 UTC m=+698.376558309" Jul 09 15:06:10.720744 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:10.720691 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:10.721468 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:10.720770 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:10.734202 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:10.734169 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:11.532106 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:11.532069 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xmtxs" Jul 09 15:06:14.504724 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:14.504663 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:14.505270 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:14.504768 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:14.517937 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:14.517905 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:06:14.546420 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:06:14.546381 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7" Jul 09 15:07:50.359216 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.359109 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55f4b7dddc-m8wgn"] Jul 09 15:07:50.362839 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.362809 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.371297 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.371271 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55f4b7dddc-m8wgn"] Jul 09 15:07:50.487051 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.487013 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bqpq\" (UniqueName: \"kubernetes.io/projected/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-kube-api-access-4bqpq\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.487257 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.487099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-tls-cert\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.588664 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.588616 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-tls-cert\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.588897 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.588714 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bqpq\" (UniqueName: \"kubernetes.io/projected/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-kube-api-access-4bqpq\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.591254 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.591178 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-tls-cert\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.597639 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.597610 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bqpq\" (UniqueName: \"kubernetes.io/projected/78c97e2c-e3ff-4369-a4ba-63bcd1c2d046-kube-api-access-4bqpq\") pod \"authorino-55f4b7dddc-m8wgn\" (UID: \"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046\") " pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.673932 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.673815 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" Jul 09 15:07:50.820341 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.820299 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55f4b7dddc-m8wgn"] Jul 09 15:07:50.823906 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:07:50.823863 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78c97e2c_e3ff_4369_a4ba_63bcd1c2d046.slice/crio-7aa1cf27e7a8417ece4373db005a217376e447f73dcb71920692d7cabbff0898 WatchSource:0}: Error finding container 7aa1cf27e7a8417ece4373db005a217376e447f73dcb71920692d7cabbff0898: Status 404 returned error can't find the container with id 7aa1cf27e7a8417ece4373db005a217376e447f73dcb71920692d7cabbff0898 Jul 09 15:07:50.991820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:50.991777 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" event={"ID":"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046","Type":"ContainerStarted","Data":"7aa1cf27e7a8417ece4373db005a217376e447f73dcb71920692d7cabbff0898"} Jul 09 15:07:53.003801 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.003761 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" event={"ID":"78c97e2c-e3ff-4369-a4ba-63bcd1c2d046","Type":"ContainerStarted","Data":"42dff8f1daf19a77f199d2be417998502d7c2e99b1520a216c2ced7d5289a507"} Jul 09 15:07:53.027434 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.027380 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55f4b7dddc-m8wgn" podStartSLOduration=0.943999742 podStartE2EDuration="3.027360694s" totalImagesPullingTime="2.083360952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:07:53.026819099 +0000 UTC m=+805.889826294" watchObservedRunningTime="2026-07-09 15:07:53.027360694 +0000 UTC m=+805.890367863" Jul 09 15:07:53.070171 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.070121 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:07:53.070723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.070674 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6df9f875c8-5cftz" podUID="1eb73c66-e43d-4541-925d-b52dbd3f6501" containerName="authorino" containerID="cri-o://0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e" gracePeriod=30 Jul 09 15:07:53.335263 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.335232 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:07:53.414028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.413992 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert\") pod \"1eb73c66-e43d-4541-925d-b52dbd3f6501\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " Jul 09 15:07:53.414185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.414092 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xkd4\" (UniqueName: \"kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4\") pod \"1eb73c66-e43d-4541-925d-b52dbd3f6501\" (UID: \"1eb73c66-e43d-4541-925d-b52dbd3f6501\") " Jul 09 15:07:53.416356 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.416320 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4" (OuterVolumeSpecName: "kube-api-access-5xkd4") pod "1eb73c66-e43d-4541-925d-b52dbd3f6501" (UID: "1eb73c66-e43d-4541-925d-b52dbd3f6501"). InnerVolumeSpecName "kube-api-access-5xkd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:07:53.426061 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.426024 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1eb73c66-e43d-4541-925d-b52dbd3f6501" (UID: "1eb73c66-e43d-4541-925d-b52dbd3f6501"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:07:53.514780 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.514743 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xkd4\" (UniqueName: \"kubernetes.io/projected/1eb73c66-e43d-4541-925d-b52dbd3f6501-kube-api-access-5xkd4\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:07:53.514780 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:53.514771 2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1eb73c66-e43d-4541-925d-b52dbd3f6501-tls-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:07:54.010741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.010707 2558 generic.go:356] "Generic (PLEG): container finished" podID="1eb73c66-e43d-4541-925d-b52dbd3f6501" containerID="0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e" exitCode=0 Jul 09 15:07:54.011185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.010766 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6df9f875c8-5cftz" Jul 09 15:07:54.011185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.010792 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df9f875c8-5cftz" event={"ID":"1eb73c66-e43d-4541-925d-b52dbd3f6501","Type":"ContainerDied","Data":"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e"} Jul 09 15:07:54.011185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.010839 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6df9f875c8-5cftz" event={"ID":"1eb73c66-e43d-4541-925d-b52dbd3f6501","Type":"ContainerDied","Data":"7e141258363f085bdd6bc31a2d78a28f35308cebdff28f909facbe90b7362c05"} Jul 09 15:07:54.011185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.010860 2558 scope.go:117] "RemoveContainer" containerID="0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e" Jul 09 15:07:54.020246 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.020222 2558 scope.go:117] "RemoveContainer" containerID="0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e" Jul 09 15:07:54.020558 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:07:54.020532 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e\": container with ID starting with 0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e not found: ID does not exist" containerID="0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e" Jul 09 15:07:54.020691 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.020562 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e"} err="failed to get container status \"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e\": rpc error: code = NotFound desc = could not find container \"0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e\": container with ID starting with 0d66f94c5a35cd8b05640ae4d0f0f15489b27e8680022ba1b2e2c38abfef3c4e not found: ID does not exist" Jul 09 15:07:54.031623 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.031588 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:07:54.035485 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:54.035457 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6df9f875c8-5cftz"] Jul 09 15:07:55.625963 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:07:55.625897 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eb73c66-e43d-4541-925d-b52dbd3f6501" path="/var/lib/kubelet/pods/1eb73c66-e43d-4541-925d-b52dbd3f6501/volumes" Jul 09 15:09:27.616715 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:09:27.616609 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:09:27.616715 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:09:27.616654 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:13:56.253165 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:56.253067 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:13:56.253654 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:56.253307 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" podUID="2937b7a9-127f-4094-9979-a44093ee32e9" containerName="manager" containerID="cri-o://f82db3ede2fe265dda8d68ba5cfe23bbf71432cb489b0398759595556bb81a64" gracePeriod=10 Jul 09 15:13:58.780844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:58.780812 2558 generic.go:356] "Generic (PLEG): container finished" podID="2937b7a9-127f-4094-9979-a44093ee32e9" containerID="f82db3ede2fe265dda8d68ba5cfe23bbf71432cb489b0398759595556bb81a64" exitCode=0 Jul 09 15:13:58.781219 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:58.780903 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" event={"ID":"2937b7a9-127f-4094-9979-a44093ee32e9","Type":"ContainerDied","Data":"f82db3ede2fe265dda8d68ba5cfe23bbf71432cb489b0398759595556bb81a64"} Jul 09 15:13:59.310943 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.310916 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:13:59.447898 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.447806 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crgwj\" (UniqueName: \"kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj\") pod \"2937b7a9-127f-4094-9979-a44093ee32e9\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " Jul 09 15:13:59.447898 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.447843 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume\") pod \"2937b7a9-127f-4094-9979-a44093ee32e9\" (UID: \"2937b7a9-127f-4094-9979-a44093ee32e9\") " Jul 09 15:13:59.448243 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.448222 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2937b7a9-127f-4094-9979-a44093ee32e9" (UID: "2937b7a9-127f-4094-9979-a44093ee32e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:13:59.449911 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.449868 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj" (OuterVolumeSpecName: "kube-api-access-crgwj") pod "2937b7a9-127f-4094-9979-a44093ee32e9" (UID: "2937b7a9-127f-4094-9979-a44093ee32e9"). InnerVolumeSpecName "kube-api-access-crgwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:13:59.548638 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.548597 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crgwj\" (UniqueName: \"kubernetes.io/projected/2937b7a9-127f-4094-9979-a44093ee32e9-kube-api-access-crgwj\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:13:59.548638 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.548631 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2937b7a9-127f-4094-9979-a44093ee32e9-extensions-socket-volume\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:13:59.786959 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.786926 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" Jul 09 15:13:59.786959 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.786941 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh" event={"ID":"2937b7a9-127f-4094-9979-a44093ee32e9","Type":"ContainerDied","Data":"f93b97e4d33395108122b6c2c1da5a21a7a9c6f4b598fecb11613b5144c69f30"} Jul 09 15:13:59.787493 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.787003 2558 scope.go:117] "RemoveContainer" containerID="f82db3ede2fe265dda8d68ba5cfe23bbf71432cb489b0398759595556bb81a64" Jul 09 15:13:59.808700 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.808671 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:13:59.815429 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:13:59.815405 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l89sh"] Jul 09 15:14:01.625200 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:01.625162 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2937b7a9-127f-4094-9979-a44093ee32e9" path="/var/lib/kubelet/pods/2937b7a9-127f-4094-9979-a44093ee32e9/volumes" Jul 09 15:14:27.643989 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:27.643961 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:14:27.645427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:27.645403 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:14:57.491074 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491040 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v"] Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491379 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb73c66-e43d-4541-925d-b52dbd3f6501" containerName="authorino" Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491390 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb73c66-e43d-4541-925d-b52dbd3f6501" containerName="authorino" Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491401 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2937b7a9-127f-4094-9979-a44093ee32e9" containerName="manager" Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491406 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2937b7a9-127f-4094-9979-a44093ee32e9" containerName="manager" Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491453 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2937b7a9-127f-4094-9979-a44093ee32e9" containerName="manager" Jul 09 15:14:57.491512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.491464 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb73c66-e43d-4541-925d-b52dbd3f6501" containerName="authorino" Jul 09 15:14:57.494280 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.494265 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.497653 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.497631 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kg9w2\"" Jul 09 15:14:57.516773 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.516738 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v"] Jul 09 15:14:57.609149 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.609110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.609323 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.609167 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5kxq\" (UniqueName: \"kubernetes.io/projected/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-kube-api-access-g5kxq\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.710205 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.710166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.710461 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.710227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5kxq\" (UniqueName: \"kubernetes.io/projected/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-kube-api-access-g5kxq\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.710597 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.710575 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.721769 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.721737 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5kxq\" (UniqueName: \"kubernetes.io/projected/7cb8cc90-5019-4bf0-b85b-37baf37d0b44-kube-api-access-g5kxq\") pod \"kuadrant-operator-controller-manager-86878c769f-lbq2v\" (UID: \"7cb8cc90-5019-4bf0-b85b-37baf37d0b44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.804059 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.803962 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:57.948134 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:57.948100 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v"] Jul 09 15:14:57.951531 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:14:57.951500 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb8cc90_5019_4bf0_b85b_37baf37d0b44.slice/crio-774656df582cdaec94667a98f58b838e897571dbf28e88fa9c8582d90768c6d9 WatchSource:0}: Error finding container 774656df582cdaec94667a98f58b838e897571dbf28e88fa9c8582d90768c6d9: Status 404 returned error can't find the container with id 774656df582cdaec94667a98f58b838e897571dbf28e88fa9c8582d90768c6d9 Jul 09 15:14:58.065978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:58.065889 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" event={"ID":"7cb8cc90-5019-4bf0-b85b-37baf37d0b44","Type":"ContainerStarted","Data":"288ae21bb41c8e06d0dd33948c7bbccbf07a516184bda3e45c9f8149af899dcb"} Jul 09 15:14:58.065978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:58.065947 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" event={"ID":"7cb8cc90-5019-4bf0-b85b-37baf37d0b44","Type":"ContainerStarted","Data":"774656df582cdaec94667a98f58b838e897571dbf28e88fa9c8582d90768c6d9"} Jul 09 15:14:58.065978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:58.065975 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:14:58.100302 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:14:58.100250 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" podStartSLOduration=1.100234008 podStartE2EDuration="1.100234008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:58.099146544 +0000 UTC m=+1230.962153711" watchObservedRunningTime="2026-07-09 15:14:58.100234008 +0000 UTC m=+1230.963241175" Jul 09 15:15:09.073734 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:15:09.073705 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbq2v" Jul 09 15:19:27.670649 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:19:27.670620 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:19:27.674298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:19:27.674274 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:20:59.490322 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.490294 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:20:59.493955 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.493935 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.498448 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.498423 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0fa0eda3-openshift-default-dockercfg-f7lhg\"" Jul 09 15:20:59.507143 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507245 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507160 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507245 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bxcx\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507245 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507291 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507313 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507338 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.507378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.507353 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.523500 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.523476 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:20:59.608394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bxcx\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608478 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608547 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608841 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608598 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608841 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608841 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608995 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608904 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.608995 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.608954 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.609067 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.609048 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.609142 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.609117 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.609246 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.609191 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.610960 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.610933 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.611285 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.611270 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.627828 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.627787 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bxcx\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.627828 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.627808 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token\") pod \"e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.804611 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.804532 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:20:59.960204 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.960174 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:20:59.961711 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:20:59.961678 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86063182_def0_491f_a10a_e91b45c868bc.slice/crio-8b4d379f9c815024ea66cde014d0685f2294f160931e2d8b55cda7428354dbbd WatchSource:0}: Error finding container 8b4d379f9c815024ea66cde014d0685f2294f160931e2d8b55cda7428354dbbd: Status 404 returned error can't find the container with id 8b4d379f9c815024ea66cde014d0685f2294f160931e2d8b55cda7428354dbbd Jul 09 15:20:59.963991 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.963953 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:20:59.964079 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.964024 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:20:59.964079 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:20:59.964055 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:00.736265 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:00.736234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" event={"ID":"86063182-def0-491f-a10a-e91b45c868bc","Type":"ContainerStarted","Data":"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d"} Jul 09 15:21:00.736652 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:00.736288 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" event={"ID":"86063182-def0-491f-a10a-e91b45c868bc","Type":"ContainerStarted","Data":"8b4d379f9c815024ea66cde014d0685f2294f160931e2d8b55cda7428354dbbd"} Jul 09 15:21:00.779325 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:00.779263 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" podStartSLOduration=1.77924287 podStartE2EDuration="1.77924287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:21:00.777407719 +0000 UTC m=+1593.640414914" watchObservedRunningTime="2026-07-09 15:21:00.77924287 +0000 UTC m=+1593.642250086" Jul 09 15:21:00.805653 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:00.805618 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:01.809752 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:01.809721 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:02.745595 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:02.745567 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:02.746535 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:02.746512 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:06.170832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:06.170796 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:21:06.171440 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:06.171074 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" podUID="86063182-def0-491f-a10a-e91b45c868bc" containerName="istio-proxy" containerID="cri-o://dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d" gracePeriod=30 Jul 09 15:21:11.323916 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.323868 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:11.427368 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427267 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427368 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427303 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427368 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427325 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bxcx\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427368 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427356 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427404 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427434 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427464 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427533 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427576 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token\") pod \"86063182-def0-491f-a10a-e91b45c868bc\" (UID: \"86063182-def0-491f-a10a-e91b45c868bc\") " Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427650 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:11.427695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427677 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:21:11.427948 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427846 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86063182-def0-491f-a10a-e91b45c868bc-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.427948 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.427859 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.430091 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.430062 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:11.430224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.430167 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:11.430509 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.430488 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:11.431869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.431839 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:21:11.432035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.431918 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx" (OuterVolumeSpecName: "kube-api-access-9bxcx") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "kube-api-access-9bxcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:11.432301 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.432276 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:11.432301 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.432290 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "86063182-def0-491f-a10a-e91b45c868bc" (UID: "86063182-def0-491f-a10a-e91b45c868bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:11.528443 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528394 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528443 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528434 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528443 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528448 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bxcx\" (UniqueName: \"kubernetes.io/projected/86063182-def0-491f-a10a-e91b45c868bc-kube-api-access-9bxcx\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528443 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528460 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86063182-def0-491f-a10a-e91b45c868bc-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528472 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528484 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.528721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.528494 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86063182-def0-491f-a10a-e91b45c868bc-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:11.795394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.795353 2558 generic.go:356] "Generic (PLEG): container finished" podID="86063182-def0-491f-a10a-e91b45c868bc" containerID="dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d" exitCode=0 Jul 09 15:21:11.795571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.795423 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" Jul 09 15:21:11.795571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.795444 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" event={"ID":"86063182-def0-491f-a10a-e91b45c868bc","Type":"ContainerDied","Data":"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d"} Jul 09 15:21:11.795571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.795528 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq" event={"ID":"86063182-def0-491f-a10a-e91b45c868bc","Type":"ContainerDied","Data":"8b4d379f9c815024ea66cde014d0685f2294f160931e2d8b55cda7428354dbbd"} Jul 09 15:21:11.795571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.795550 2558 scope.go:117] "RemoveContainer" containerID="dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d" Jul 09 15:21:11.804521 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.804505 2558 scope.go:117] "RemoveContainer" containerID="dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d" Jul 09 15:21:11.804782 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:21:11.804760 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d\": container with ID starting with dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d not found: ID does not exist" containerID="dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d" Jul 09 15:21:11.804832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.804791 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d"} err="failed to get container status \"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d\": rpc error: code = NotFound desc = could not find container \"dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d\": container with ID starting with dfa54527146e350e3e73ba51f493b91277a93bfbb1a56832ea484bec532b043d not found: ID does not exist" Jul 09 15:21:11.824544 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.824515 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:21:11.830081 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:11.830059 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0fa0eda3-openshift-default-576f845c87-xzslq"] Jul 09 15:21:12.542828 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.542791 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:12.543345 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.543159 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86063182-def0-491f-a10a-e91b45c868bc" containerName="istio-proxy" Jul 09 15:21:12.543345 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.543171 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86063182-def0-491f-a10a-e91b45c868bc" containerName="istio-proxy" Jul 09 15:21:12.543345 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.543266 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86063182-def0-491f-a10a-e91b45c868bc" containerName="istio-proxy" Jul 09 15:21:12.551469 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.551446 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.554926 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.554908 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0fc76beb-openshift-default-dockercfg-zglxl\"" Jul 09 15:21:12.560263 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.560205 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:12.638644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.638644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638651 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.638905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638682 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.638905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638803 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.638988 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.638988 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638960 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l5gm\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.639075 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.638996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.639075 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.639041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.639075 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.639062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.739747 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739701 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.739747 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l5gm\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739853 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739894 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739930 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.739986 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740046 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740388 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740657 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740751 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740675 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.740812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.740745 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.742115 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.742094 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.742715 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.742689 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.753204 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.753182 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l5gm\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.753631 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.753610 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token\") pod \"e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:12.864502 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:12.864414 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:13.008615 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.008591 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:13.009751 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:21:13.009727 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d8b6c7_d437_4e76_b891_015cf1f92dae.slice/crio-8e5e3b37a60d0b7db770647d9969a9aa08e7b217ac60504f69cad2f2b25fe2c2 WatchSource:0}: Error finding container 8e5e3b37a60d0b7db770647d9969a9aa08e7b217ac60504f69cad2f2b25fe2c2: Status 404 returned error can't find the container with id 8e5e3b37a60d0b7db770647d9969a9aa08e7b217ac60504f69cad2f2b25fe2c2 Jul 09 15:21:13.011923 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.011893 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:13.012020 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.011957 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:13.012020 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.011989 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:13.623829 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.623795 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86063182-def0-491f-a10a-e91b45c868bc" path="/var/lib/kubelet/pods/86063182-def0-491f-a10a-e91b45c868bc/volumes" Jul 09 15:21:13.808172 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.808139 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerStarted","Data":"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc"} Jul 09 15:21:13.808339 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.808188 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerStarted","Data":"8e5e3b37a60d0b7db770647d9969a9aa08e7b217ac60504f69cad2f2b25fe2c2"} Jul 09 15:21:13.840912 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.840844 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" podStartSLOduration=1.840825736 podStartE2EDuration="1.840825736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:21:13.837995825 +0000 UTC m=+1606.701002994" watchObservedRunningTime="2026-07-09 15:21:13.840825736 +0000 UTC m=+1606.703832905" Jul 09 15:21:13.864701 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:13.864667 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:14.870375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:14.870344 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:15.818337 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:15.818307 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:15.819362 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:15.819342 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:24.864819 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.864778 2558 generic.go:356] "Generic (PLEG): container finished" podID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerID="52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc" exitCode=0 Jul 09 15:21:24.865239 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.864848 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerDied","Data":"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc"} Jul 09 15:21:24.865323 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.865256 2558 scope.go:117] "RemoveContainer" containerID="52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc" Jul 09 15:21:24.866461 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.866429 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:24.866538 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.866523 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:24.866586 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:24.866573 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:25.179807 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:25.179720 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:25.871723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:25.871684 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerStarted","Data":"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c"} Jul 09 15:21:25.872200 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:25.871793 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" containerID="cri-o://3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c" gracePeriod=30 Jul 09 15:21:30.819224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:30.819185 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:31.134809 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.134783 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:31.209132 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209101 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l5gm\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209304 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209143 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209304 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209161 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209304 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209186 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209304 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209254 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209304 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209282 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209566 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209331 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209566 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209355 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209566 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209397 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token\") pod \"08d8b6c7-d437-4e76-b891-015cf1f92dae\" (UID: \"08d8b6c7-d437-4e76-b891-015cf1f92dae\") " Jul 09 15:21:31.209566 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209405 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:31.209782 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209571 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:31.209782 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209657 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.209782 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209675 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.209950 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209783 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:31.209950 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.209690 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:21:31.210440 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.210412 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data" (OuterVolumeSpecName: "istio-data") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:31.211560 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.211529 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:31.211560 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.211540 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token" (OuterVolumeSpecName: "istio-token") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:31.211724 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.211583 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm" (OuterVolumeSpecName: "kube-api-access-8l5gm") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "kube-api-access-8l5gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:31.211812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.211786 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08d8b6c7-d437-4e76-b891-015cf1f92dae" (UID: "08d8b6c7-d437-4e76-b891-015cf1f92dae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:21:31.310489 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310456 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310489 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310482 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l5gm\" (UniqueName: \"kubernetes.io/projected/08d8b6c7-d437-4e76-b891-015cf1f92dae-kube-api-access-8l5gm\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310489 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310491 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310501 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310509 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d8b6c7-d437-4e76-b891-015cf1f92dae-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310517 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d8b6c7-d437-4e76-b891-015cf1f92dae-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.310721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.310527 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d8b6c7-d437-4e76-b891-015cf1f92dae-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:31.503985 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.503954 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:31.504376 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504359 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.504461 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504379 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.504461 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504394 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.504461 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504401 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.504666 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504493 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.504666 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.504506 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerName="istio-proxy" Jul 09 15:21:31.507937 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.507916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.512460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.512441 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-69810dc0-openshift-default-dockercfg-jm9qk\"" Jul 09 15:21:31.525116 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.525087 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:31.613741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.613741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613750 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613861 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613907 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr7ml\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613935 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613953 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614024 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.613975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.614338 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.614039 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.714745 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.714745 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714752 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714852 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714908 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714933 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714951 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr7ml\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715001 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.714979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715258 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715258 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715258 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715213 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715258 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715231 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715438 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715518 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715473 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.715726 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.715703 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.717356 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.717337 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.717414 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.717397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.725147 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.725125 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.725360 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.725344 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr7ml\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml\") pod \"e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.819604 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.819508 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:31.908458 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.908120 2558 generic.go:356] "Generic (PLEG): container finished" podID="08d8b6c7-d437-4e76-b891-015cf1f92dae" containerID="3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c" exitCode=0 Jul 09 15:21:31.908458 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.908221 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" Jul 09 15:21:31.908458 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.908250 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerDied","Data":"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c"} Jul 09 15:21:31.908458 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.908299 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2" event={"ID":"08d8b6c7-d437-4e76-b891-015cf1f92dae","Type":"ContainerDied","Data":"8e5e3b37a60d0b7db770647d9969a9aa08e7b217ac60504f69cad2f2b25fe2c2"} Jul 09 15:21:31.908458 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.908328 2558 scope.go:117] "RemoveContainer" containerID="3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c" Jul 09 15:21:31.919322 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.919294 2558 scope.go:117] "RemoveContainer" containerID="52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc" Jul 09 15:21:31.927624 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.927602 2558 scope.go:117] "RemoveContainer" containerID="3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c" Jul 09 15:21:31.927982 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:21:31.927959 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c\": container with ID starting with 3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c not found: ID does not exist" containerID="3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c" Jul 09 15:21:31.928080 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.927993 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c"} err="failed to get container status \"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c\": rpc error: code = NotFound desc = could not find container \"3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c\": container with ID starting with 3bda807089686ec0609ac85e8149d0eed2b6cd6b3cc096ade05e6cc0bc329d5c not found: ID does not exist" Jul 09 15:21:31.928080 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.928016 2558 scope.go:117] "RemoveContainer" containerID="52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc" Jul 09 15:21:31.928434 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:21:31.928288 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc\": container with ID starting with 52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc not found: ID does not exist" containerID="52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc" Jul 09 15:21:31.928434 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.928318 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc"} err="failed to get container status \"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc\": rpc error: code = NotFound desc = could not find container \"52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc\": container with ID starting with 52f519aa96c2cd0884a608c87fba76ed8dbbfa0188f484cd13b04badc3cf6afc not found: ID does not exist" Jul 09 15:21:31.930497 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.930475 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:31.935476 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.935441 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0fc76beb-openshift-default-6d5fbdbd5c-9gjz2"] Jul 09 15:21:31.997818 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:31.997603 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:32.000437 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:21:32.000404 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081c47f2_f3b9_4e98_951f_d5b7f5dae9c9.slice/crio-a1c6d78e90fd4860c9c802ec40425e2e28773af0f2561f2d6056045cebc73101 WatchSource:0}: Error finding container a1c6d78e90fd4860c9c802ec40425e2e28773af0f2561f2d6056045cebc73101: Status 404 returned error can't find the container with id a1c6d78e90fd4860c9c802ec40425e2e28773af0f2561f2d6056045cebc73101 Jul 09 15:21:32.002737 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.002707 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:32.002822 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.002774 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:32.002822 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.002807 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:21:32.915585 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.915542 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" event={"ID":"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9","Type":"ContainerStarted","Data":"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045"} Jul 09 15:21:32.916101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.915606 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" event={"ID":"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9","Type":"ContainerStarted","Data":"a1c6d78e90fd4860c9c802ec40425e2e28773af0f2561f2d6056045cebc73101"} Jul 09 15:21:32.946352 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:32.946302 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" podStartSLOduration=1.946284793 podStartE2EDuration="1.946284793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:21:32.943947853 +0000 UTC m=+1625.806955022" watchObservedRunningTime="2026-07-09 15:21:32.946284793 +0000 UTC m=+1625.809292033" Jul 09 15:21:33.629798 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:33.629759 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d8b6c7-d437-4e76-b891-015cf1f92dae" path="/var/lib/kubelet/pods/08d8b6c7-d437-4e76-b891-015cf1f92dae/volumes" Jul 09 15:21:33.819933 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:33.819870 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:33.825003 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:33.824978 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:33.920203 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:33.920122 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:33.920981 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:33.920962 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:37.329258 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:37.329221 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:37.329751 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:37.329418 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" podUID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" containerName="istio-proxy" containerID="cri-o://317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045" gracePeriod=30 Jul 09 15:21:44.190221 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.190194 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:44.227465 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227434 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.227730 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227705 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.227839 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227748 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.227839 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227776 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.227839 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227808 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.228043 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227907 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.228043 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227937 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.228043 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.227971 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.228043 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.228014 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr7ml\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml\") pod \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\" (UID: \"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9\") " Jul 09 15:21:44.228596 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.228298 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:44.228716 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.228670 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:21:44.228716 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.228688 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:44.229094 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.229068 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:44.229451 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.229428 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:44.230399 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.230374 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:44.230697 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.230636 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:21:44.230805 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.230780 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:21:44.230862 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.230808 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml" (OuterVolumeSpecName: "kube-api-access-fr7ml") pod "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" (UID: "081c47f2-f3b9-4e98-951f-d5b7f5dae9c9"). InnerVolumeSpecName "kube-api-access-fr7ml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:21:44.329427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329391 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr7ml\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-kube-api-access-fr7ml\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329423 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329436 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329449 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329463 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329474 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329484 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329496 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.329677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.329506 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:21:44.982010 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.981975 2558 generic.go:356] "Generic (PLEG): container finished" podID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" containerID="317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045" exitCode=0 Jul 09 15:21:44.982222 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.982030 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" Jul 09 15:21:44.982222 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.982062 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" event={"ID":"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9","Type":"ContainerDied","Data":"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045"} Jul 09 15:21:44.982222 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.982156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr" event={"ID":"081c47f2-f3b9-4e98-951f-d5b7f5dae9c9","Type":"ContainerDied","Data":"a1c6d78e90fd4860c9c802ec40425e2e28773af0f2561f2d6056045cebc73101"} Jul 09 15:21:44.982222 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.982188 2558 scope.go:117] "RemoveContainer" containerID="317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045" Jul 09 15:21:44.991755 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.991740 2558 scope.go:117] "RemoveContainer" containerID="317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045" Jul 09 15:21:44.992072 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:21:44.992049 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045\": container with ID starting with 317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045 not found: ID does not exist" containerID="317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045" Jul 09 15:21:44.992164 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:44.992078 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045"} err="failed to get container status \"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045\": rpc error: code = NotFound desc = could not find container \"317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045\": container with ID starting with 317e894ebdbcecd00e3a5262e3909c71f7221bb64c5cbc1998562cdcad7f0045 not found: ID does not exist" Jul 09 15:21:45.006480 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:45.006454 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:45.010455 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:45.010432 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-69810dc0-openshift-default-6c7fdb7fc5-fwjpr"] Jul 09 15:21:45.624290 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:21:45.624257 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" path="/var/lib/kubelet/pods/081c47f2-f3b9-4e98-951f-d5b7f5dae9c9/volumes" Jul 09 15:23:22.287982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.287903 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:22.288420 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.288260 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" containerName="istio-proxy" Jul 09 15:23:22.288420 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.288271 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" containerName="istio-proxy" Jul 09 15:23:22.288420 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.288344 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="081c47f2-f3b9-4e98-951f-d5b7f5dae9c9" containerName="istio-proxy" Jul 09 15:23:22.291214 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.291197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.295185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.295158 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8d76c514-openshift-default-dockercfg-kng28\"" Jul 09 15:23:22.335256 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.335220 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:22.475661 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475721 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475784 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475812 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.475869 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475905 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9n75\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.476182 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.475982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.476182 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.476032 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.576982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.576858 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.576982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.576942 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.576982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.576965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.576982 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.576992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577178 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577318 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577253 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9n75\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577564 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577462 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577564 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577513 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.577564 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577526 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.578115 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.577698 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.578241 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.578220 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.579335 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.579310 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.579456 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.579439 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.587226 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.587201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9n75\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.587338 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.587208 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token\") pod \"e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.601206 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.601184 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:22.746526 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.746494 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:22.748036 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:23:22.748007 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a98177_c317_413e_b078_d0bc10d3f7ef.slice/crio-01736e50d155eb9501ffaa47622cd2965d105c8b1987f98bf96111765338e7b6 WatchSource:0}: Error finding container 01736e50d155eb9501ffaa47622cd2965d105c8b1987f98bf96111765338e7b6: Status 404 returned error can't find the container with id 01736e50d155eb9501ffaa47622cd2965d105c8b1987f98bf96111765338e7b6 Jul 09 15:23:22.750215 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.750183 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:22.750297 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.750248 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:22.750297 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:22.750277 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:23.456184 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:23.456144 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerStarted","Data":"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4"} Jul 09 15:23:23.456575 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:23.456194 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerStarted","Data":"01736e50d155eb9501ffaa47622cd2965d105c8b1987f98bf96111765338e7b6"} Jul 09 15:23:23.482338 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:23.482281 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podStartSLOduration=1.482262408 podStartE2EDuration="1.482262408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:23:23.481782138 +0000 UTC m=+1736.344789311" watchObservedRunningTime="2026-07-09 15:23:23.482262408 +0000 UTC m=+1736.345269577" Jul 09 15:23:23.602222 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:23.602187 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:23.607117 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:23.607086 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:24.460624 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:24.460595 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:24.461509 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:24.461492 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:39.462041 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.462006 2558 patch_prober.go:28] interesting pod/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 09 15:23:39.462460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.462089 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 09 15:23:39.531326 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.531293 2558 generic.go:356] "Generic (PLEG): container finished" podID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerID="0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4" exitCode=0 Jul 09 15:23:39.531488 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.531334 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerDied","Data":"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4"} Jul 09 15:23:39.531686 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.531673 2558 scope.go:117] "RemoveContainer" containerID="0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4" Jul 09 15:23:39.532660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.532632 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:39.532726 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.532702 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:39.532766 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:39.532734 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:23:40.539122 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:40.539073 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerStarted","Data":"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f"} Jul 09 15:23:40.601704 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:40.601665 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:40.601940 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:40.601917 2558 patch_prober.go:28] interesting pod/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 09 15:23:40.601990 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:40.601971 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 09 15:23:41.601612 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:41.601578 2558 patch_prober.go:28] interesting pod/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 09 15:23:41.602032 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:41.601641 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 09 15:23:42.602168 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:42.602132 2558 patch_prober.go:28] interesting pod/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 09 15:23:42.602547 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:42.602189 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 09 15:23:43.601617 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:43.601585 2558 patch_prober.go:28] interesting pod/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 09 15:23:43.601802 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:43.601637 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 09 15:23:44.605847 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:44.605817 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:44.606254 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:44.606109 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:44.606864 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:44.606844 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:44.844052 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:44.844011 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:45.568730 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:45.568695 2558 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" secret="" err="secret \"e2e-mt-8d76c514-openshift-default-dockercfg-kng28\" not found" Jul 09 15:23:46.573231 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:46.573166 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" containerID="cri-o://8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f" gracePeriod=30 Jul 09 15:23:51.818351 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.818327 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:51.946062 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.945965 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946062 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946023 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946062 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946063 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946091 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9n75\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946118 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946143 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946174 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946208 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946252 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy\") pod \"41a98177-c317-413e-b078-d0bc10d3f7ef\" (UID: \"41a98177-c317-413e-b078-d0bc10d3f7ef\") " Jul 09 15:23:51.946373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946285 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:51.946723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946504 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:51.946723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946524 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:51.946723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946538 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:23:51.946723 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946689 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:51.947011 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.946986 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:51.948367 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.948339 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:23:51.948482 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.948454 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:23:51.948482 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.948454 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:23:51.948482 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:51.948473 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75" (OuterVolumeSpecName: "kube-api-access-j9n75") pod "41a98177-c317-413e-b078-d0bc10d3f7ef" (UID: "41a98177-c317-413e-b078-d0bc10d3f7ef"). InnerVolumeSpecName "kube-api-access-j9n75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:23:52.047233 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047194 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047233 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047224 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047233 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047235 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047233 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047243 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047514 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047251 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9n75\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-kube-api-access-j9n75\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047514 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047261 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a98177-c317-413e-b078-d0bc10d3f7ef-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047514 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047269 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a98177-c317-413e-b078-d0bc10d3f7ef-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.047514 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.047277 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a98177-c317-413e-b078-d0bc10d3f7ef-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:23:52.603147 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.603115 2558 generic.go:356] "Generic (PLEG): container finished" podID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerID="8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f" exitCode=0 Jul 09 15:23:52.603357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.603165 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" Jul 09 15:23:52.603357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.603201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerDied","Data":"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f"} Jul 09 15:23:52.603357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.603260 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh" event={"ID":"41a98177-c317-413e-b078-d0bc10d3f7ef","Type":"ContainerDied","Data":"01736e50d155eb9501ffaa47622cd2965d105c8b1987f98bf96111765338e7b6"} Jul 09 15:23:52.603357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.603302 2558 scope.go:117] "RemoveContainer" containerID="8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f" Jul 09 15:23:52.612238 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.612220 2558 scope.go:117] "RemoveContainer" containerID="0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4" Jul 09 15:23:52.620365 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.620347 2558 scope.go:117] "RemoveContainer" containerID="8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f" Jul 09 15:23:52.620618 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:23:52.620597 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f\": container with ID starting with 8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f not found: ID does not exist" containerID="8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f" Jul 09 15:23:52.620703 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.620623 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f"} err="failed to get container status \"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f\": rpc error: code = NotFound desc = could not find container \"8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f\": container with ID starting with 8357559e03ec5d9f88b385fdededf38046e2ca2af87ef3b38adbd206e357143f not found: ID does not exist" Jul 09 15:23:52.620703 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.620647 2558 scope.go:117] "RemoveContainer" containerID="0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4" Jul 09 15:23:52.620827 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:23:52.620811 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4\": container with ID starting with 0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4 not found: ID does not exist" containerID="0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4" Jul 09 15:23:52.620900 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.620832 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4"} err="failed to get container status \"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4\": rpc error: code = NotFound desc = could not find container \"0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4\": container with ID starting with 0313a2a9afffa1323ecd6db46f0dc604e843038f148e4b688ebe233bb2e420c4 not found: ID does not exist" Jul 09 15:23:52.625980 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.625958 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:52.630359 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:52.630339 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8d76c514-openshift-default-65c55b8cc9-k42vh"] Jul 09 15:23:53.623665 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:23:53.623627 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" path="/var/lib/kubelet/pods/41a98177-c317-413e-b078-d0bc10d3f7ef/volumes" Jul 09 15:24:05.719754 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.716242 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:24:05.719754 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.716714 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:24:05.719754 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.716728 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:24:05.719754 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.716824 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:24:05.721053 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.721028 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.725453 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.725432 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-eebed0bf-openshift-default-dockercfg-x8cdb\"" Jul 09 15:24:05.733549 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.733513 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:24:05.872845 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.872804 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873050 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.872928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873050 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.872972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873050 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.872994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873050 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.873011 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.873092 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.873154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.873198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.873224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.873221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqp2f\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974182 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974317 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974344 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.974494 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.974377 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqp2f\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.975036 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.975005 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.975148 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.975032 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.975148 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.975014 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.975148 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.975049 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.975317 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.975181 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.976791 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.976770 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.977351 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.977332 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.983511 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.983488 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:05.983644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:05.983626 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqp2f\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f\") pod \"e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:06.039811 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.039762 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:06.196264 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.196107 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:24:06.198571 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:24:06.198543 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6288258_81b9_42c0_bcc2_fa1eaee44ab6.slice/crio-f7cfb2e63448a6860a64b3b31351f99a6839f95e84ada348f5da367c120aca0a WatchSource:0}: Error finding container f7cfb2e63448a6860a64b3b31351f99a6839f95e84ada348f5da367c120aca0a: Status 404 returned error can't find the container with id f7cfb2e63448a6860a64b3b31351f99a6839f95e84ada348f5da367c120aca0a Jul 09 15:24:06.201004 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.200962 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:06.201114 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.201055 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:06.201114 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.201095 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:06.676610 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.676525 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerStarted","Data":"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0"} Jul 09 15:24:06.676610 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.676591 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerStarted","Data":"f7cfb2e63448a6860a64b3b31351f99a6839f95e84ada348f5da367c120aca0a"} Jul 09 15:24:06.698927 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:06.698860 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podStartSLOduration=1.6988443370000001 podStartE2EDuration="1.698844337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:24:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:24:06.696924364 +0000 UTC m=+1779.559931553" watchObservedRunningTime="2026-07-09 15:24:06.698844337 +0000 UTC m=+1779.561851505" Jul 09 15:24:07.040279 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:07.040241 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:07.042035 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:07.042004 2558 patch_prober.go:28] interesting pod/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 09 15:24:07.042180 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:07.042067 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 09 15:24:08.045289 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:08.045258 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:08.686653 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:08.686573 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:08.687512 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:08.687495 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:22.752241 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.752201 2558 generic.go:356] "Generic (PLEG): container finished" podID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerID="7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0" exitCode=0 Jul 09 15:24:22.752820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.752267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerDied","Data":"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0"} Jul 09 15:24:22.752820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.752765 2558 scope.go:117] "RemoveContainer" containerID="7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0" Jul 09 15:24:22.753859 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.753833 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:22.753953 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.753913 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:22.753995 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:22.753953 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:24:23.686589 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:23.686548 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:23.762998 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:23.762954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerStarted","Data":"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6"} Jul 09 15:24:24.040853 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:24.040804 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:24.042369 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:24.042336 2558 patch_prober.go:28] interesting pod/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 09 15:24:24.042499 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:24.042396 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 09 15:24:25.040935 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:25.040896 2558 patch_prober.go:28] interesting pod/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 09 15:24:25.041519 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:25.041485 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 09 15:24:26.048401 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:26.048346 2558 patch_prober.go:28] interesting pod/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 09 15:24:26.048923 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:26.048431 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 09 15:24:27.041591 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:27.041147 2558 patch_prober.go:28] interesting pod/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 09 15:24:27.041591 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:27.041210 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 09 15:24:27.698161 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:27.698131 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:24:27.702349 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:27.702327 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:24:28.044741 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:28.044706 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:28.045023 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:28.044999 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:24:28.045757 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:24:28.045740 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:25:51.612137 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:51.612044 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:25:51.612624 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:51.612365 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" containerID="cri-o://36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6" gracePeriod=30 Jul 09 15:25:56.902844 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:56.902819 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:25:57.000129 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000097 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000129 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000133 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqp2f\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000171 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000202 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000218 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000244 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000289 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000336 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000363 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data\") pod \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\" (UID: \"b6288258-81b9-42c0-bcc2-fa1eaee44ab6\") " Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000509 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000520 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000537 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000638 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000657 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.000735 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000672 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.001076 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.000805 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:25:57.001453 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.001422 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:25:57.002630 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.002600 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:25:57.002737 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.002651 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:25:57.002737 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.002705 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f" (OuterVolumeSpecName: "kube-api-access-tqp2f") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "kube-api-access-tqp2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:25:57.002858 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.002840 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6288258-81b9-42c0-bcc2-fa1eaee44ab6" (UID: "b6288258-81b9-42c0-bcc2-fa1eaee44ab6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:25:57.101721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101687 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.101721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101715 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.101721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101724 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.101721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101732 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.102000 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101740 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqp2f\" (UniqueName: \"kubernetes.io/projected/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-kube-api-access-tqp2f\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.102000 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.101750 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6288258-81b9-42c0-bcc2-fa1eaee44ab6-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:25:57.197485 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.197400 2558 generic.go:356] "Generic (PLEG): container finished" podID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerID="36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6" exitCode=0 Jul 09 15:25:57.197485 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.197447 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" Jul 09 15:25:57.197485 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.197478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerDied","Data":"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6"} Jul 09 15:25:57.197692 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.197530 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6" event={"ID":"b6288258-81b9-42c0-bcc2-fa1eaee44ab6","Type":"ContainerDied","Data":"f7cfb2e63448a6860a64b3b31351f99a6839f95e84ada348f5da367c120aca0a"} Jul 09 15:25:57.197692 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.197553 2558 scope.go:117] "RemoveContainer" containerID="36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6" Jul 09 15:25:57.207067 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.207050 2558 scope.go:117] "RemoveContainer" containerID="7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0" Jul 09 15:25:57.217245 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.217219 2558 scope.go:117] "RemoveContainer" containerID="36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6" Jul 09 15:25:57.217493 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:25:57.217470 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6\": container with ID starting with 36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6 not found: ID does not exist" containerID="36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6" Jul 09 15:25:57.217571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.217500 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6"} err="failed to get container status \"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6\": rpc error: code = NotFound desc = could not find container \"36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6\": container with ID starting with 36fdb853c2d416198c89421c45f2a871528bd38f600459737f1969ef0f9a53b6 not found: ID does not exist" Jul 09 15:25:57.217571 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.217517 2558 scope.go:117] "RemoveContainer" containerID="7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0" Jul 09 15:25:57.217771 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:25:57.217753 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0\": container with ID starting with 7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0 not found: ID does not exist" containerID="7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0" Jul 09 15:25:57.217835 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.217777 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0"} err="failed to get container status \"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0\": rpc error: code = NotFound desc = could not find container \"7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0\": container with ID starting with 7241331764ddf4bbd4e9419ffaa0e342789f1a770d47a013d3dcdc7681eb2ef0 not found: ID does not exist" Jul 09 15:25:57.226664 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.226639 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:25:57.231445 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.231423 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-eebed0bf-openshift-default-589dcc795-xjxz6"] Jul 09 15:25:57.624378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:25:57.624346 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" path="/var/lib/kubelet/pods/b6288258-81b9-42c0-bcc2-fa1eaee44ab6/volumes" Jul 09 15:26:49.017655 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.017621 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018020 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018031 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018046 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018052 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018061 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.018088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018067 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.018303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018120 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a98177-c317-413e-b078-d0bc10d3f7ef" containerName="istio-proxy" Jul 09 15:26:49.018303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018130 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.018303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.018136 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6288258-81b9-42c0-bcc2-fa1eaee44ab6" containerName="istio-proxy" Jul 09 15:26:49.021074 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.021058 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.026108 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.026089 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-476308-openshift-default-dockercfg-cz47x\"" Jul 09 15:26:49.055807 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.055780 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:26:49.144094 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144056 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144094 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144091 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144234 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144253 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144290 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144455 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144320 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.144455 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.144400 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc5rg\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245163 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245190 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245251 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245298 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245322 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245442 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245766 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245450 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc5rg\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.245766 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.245605 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.246167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.246141 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.246330 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.246181 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.246330 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.246310 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.246504 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.246479 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.247766 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.247744 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.248101 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.248085 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.261820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.261798 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.264477 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.264459 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc5rg\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg\") pod \"e2e-inf-a-476308-openshift-default-55687f9977-rzd9r\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.331375 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.331289 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:49.476149 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.476126 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:26:49.477966 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:26:49.477936 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd225c2b2_b440_4bed_b6df_f4eaa80fb8f4.slice/crio-e6238905f51a0caff61024053f5738b2d20280b7428d926a723de09cd2090060 WatchSource:0}: Error finding container e6238905f51a0caff61024053f5738b2d20280b7428d926a723de09cd2090060: Status 404 returned error can't find the container with id e6238905f51a0caff61024053f5738b2d20280b7428d926a723de09cd2090060 Jul 09 15:26:49.479898 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.479848 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:26:49.479978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.479926 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:26:49.479978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:49.479957 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:26:50.457581 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:50.457551 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerStarted","Data":"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f"} Jul 09 15:26:50.457983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:50.457601 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerStarted","Data":"e6238905f51a0caff61024053f5738b2d20280b7428d926a723de09cd2090060"} Jul 09 15:26:50.487341 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:50.487290 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podStartSLOduration=2.487269811 podStartE2EDuration="2.487269811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:50.485443243 +0000 UTC m=+1943.348450412" watchObservedRunningTime="2026-07-09 15:26:50.487269811 +0000 UTC m=+1943.350276981" Jul 09 15:26:51.331427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:51.331393 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:51.336252 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:51.336228 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:51.462525 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:51.462492 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:26:51.463390 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:26:51.463374 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:27:07.504113 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:07.504067 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 15:27:07.504677 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:07.504175 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 15:27:08.545091 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.545057 2558 generic.go:356] "Generic (PLEG): container finished" podID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerID="b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f" exitCode=0 Jul 09 15:27:08.545624 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.545143 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerDied","Data":"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f"} Jul 09 15:27:08.545624 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.545582 2558 scope.go:117] "RemoveContainer" containerID="b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f" Jul 09 15:27:08.546581 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.546554 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:27:08.546681 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.546622 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:27:08.546681 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:08.546652 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:27:09.556066 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:09.556031 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerStarted","Data":"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984"} Jul 09 15:27:10.332188 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:10.332150 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:27:10.332416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:10.332395 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 09 15:27:10.332465 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:10.332445 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 09 15:27:11.331894 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:11.331843 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 09 15:27:11.332308 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:11.331920 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 09 15:27:12.332401 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:12.332358 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 09 15:27:12.332930 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:12.332443 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 09 15:27:13.331970 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:13.331927 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 09 15:27:13.332180 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:13.331995 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 09 15:27:14.336070 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:14.336038 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:27:14.336531 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:14.336328 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:27:14.337049 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:14.337032 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:27:52.611058 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.611027 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:27:52.613426 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.613410 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.617140 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.617093 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da3a23\"/\"openshift-service-ca.crt\"" Jul 09 15:27:52.618696 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.618676 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da3a23\"/\"default-dockercfg-gtmxq\"" Jul 09 15:27:52.618821 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.618683 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-da3a23\"/\"kube-root-ca.crt\"" Jul 09 15:27:52.618821 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.618811 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-da3a23\"/\"test-model-da3a23-kserve-self-signed-certs\"" Jul 09 15:27:52.626609 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.626585 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:27:52.691501 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691469 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcvxw\" (UniqueName: \"kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.691694 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.691694 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691591 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.691694 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691607 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.691900 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691699 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.691900 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.691729 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792721 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792732 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792749 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792769 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcvxw\" (UniqueName: \"kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.792905 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.792864 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.793236 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.793206 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.793303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.793239 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.793303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.793271 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.794915 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.794893 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.795260 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.795242 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.803635 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.803614 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcvxw\" (UniqueName: \"kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw\") pod \"test-model-da3a23-kserve-867c5bc47c-nrr5d\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:52.923194 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:52.923096 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:27:53.059339 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:53.059319 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:27:53.061377 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:27:53.061350 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91fa8976_08c1_4c1e_bac1_08f597dd155e.slice/crio-d79dc9e1dc38421600695d47d78ca86e084510afc382a62869fc9961b7206a12 WatchSource:0}: Error finding container d79dc9e1dc38421600695d47d78ca86e084510afc382a62869fc9961b7206a12: Status 404 returned error can't find the container with id d79dc9e1dc38421600695d47d78ca86e084510afc382a62869fc9961b7206a12 Jul 09 15:27:53.769486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:53.769450 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" event={"ID":"91fa8976-08c1-4c1e-bac1-08f597dd155e","Type":"ContainerStarted","Data":"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d"} Jul 09 15:27:53.769983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:53.769497 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" event={"ID":"91fa8976-08c1-4c1e-bac1-08f597dd155e","Type":"ContainerStarted","Data":"d79dc9e1dc38421600695d47d78ca86e084510afc382a62869fc9961b7206a12"} Jul 09 15:27:53.795208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:27:53.795143 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" podStartSLOduration=1.795121787 podStartE2EDuration="1.795121787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:53.792342392 +0000 UTC m=+2006.655349592" watchObservedRunningTime="2026-07-09 15:27:53.795121787 +0000 UTC m=+2006.658128957" Jul 09 15:28:02.923806 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:02.923763 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:02.924378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:02.923819 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:02.936786 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:02.936760 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:03.825942 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:03.825915 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:15.624368 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:15.624338 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:28:30.648655 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.648616 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" podUID="91fa8976-08c1-4c1e-bac1-08f597dd155e" containerName="main" containerID="cri-o://51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d" gracePeriod=45 Jul 09 15:28:30.894725 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.894700 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:30.928276 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928211 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928276 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928268 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928306 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928336 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928381 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcvxw\" (UniqueName: \"kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928460 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928429 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs\") pod \"91fa8976-08c1-4c1e-bac1-08f597dd155e\" (UID: \"91fa8976-08c1-4c1e-bac1-08f597dd155e\") " Jul 09 15:28:30.928666 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928538 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:28:30.928666 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928556 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home" (OuterVolumeSpecName: "home") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:28:30.928777 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928660 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache" (OuterVolumeSpecName: "model-cache") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:28:30.928777 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928721 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-tmp-dir\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:30.928777 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.928740 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-home\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:30.930555 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.930527 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:28:30.930676 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.930527 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm" (OuterVolumeSpecName: "dshm") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:28:30.930676 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.930583 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw" (OuterVolumeSpecName: "kube-api-access-hcvxw") pod "91fa8976-08c1-4c1e-bac1-08f597dd155e" (UID: "91fa8976-08c1-4c1e-bac1-08f597dd155e"). InnerVolumeSpecName "kube-api-access-hcvxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:28:30.938992 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.938966 2558 generic.go:356] "Generic (PLEG): container finished" podID="91fa8976-08c1-4c1e-bac1-08f597dd155e" containerID="51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d" exitCode=0 Jul 09 15:28:30.939119 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.939018 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" Jul 09 15:28:30.939119 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.939024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" event={"ID":"91fa8976-08c1-4c1e-bac1-08f597dd155e","Type":"ContainerDied","Data":"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d"} Jul 09 15:28:30.939119 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.939063 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d" event={"ID":"91fa8976-08c1-4c1e-bac1-08f597dd155e","Type":"ContainerDied","Data":"d79dc9e1dc38421600695d47d78ca86e084510afc382a62869fc9961b7206a12"} Jul 09 15:28:30.939119 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.939080 2558 scope.go:117] "RemoveContainer" containerID="51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d" Jul 09 15:28:30.957482 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.957463 2558 scope.go:117] "RemoveContainer" containerID="51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d" Jul 09 15:28:30.957721 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:28:30.957704 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d\": container with ID starting with 51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d not found: ID does not exist" containerID="51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d" Jul 09 15:28:30.957763 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.957730 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d"} err="failed to get container status \"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d\": rpc error: code = NotFound desc = could not find container \"51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d\": container with ID starting with 51a391e8cf7e20c4dcccdddd98dedc77b7faadbe1eb353cf20b4cddb0a3db06d not found: ID does not exist" Jul 09 15:28:30.969471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.969450 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:28:30.973128 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:30.973108 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-da3a23/test-model-da3a23-kserve-867c5bc47c-nrr5d"] Jul 09 15:28:31.029400 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:31.029376 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-model-cache\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:31.029400 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:31.029398 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcvxw\" (UniqueName: \"kubernetes.io/projected/91fa8976-08c1-4c1e-bac1-08f597dd155e-kube-api-access-hcvxw\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:31.029546 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:31.029410 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91fa8976-08c1-4c1e-bac1-08f597dd155e-tls-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:31.029546 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:31.029419 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91fa8976-08c1-4c1e-bac1-08f597dd155e-dshm\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:28:31.624987 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:31.624954 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91fa8976-08c1-4c1e-bac1-08f597dd155e" path="/var/lib/kubelet/pods/91fa8976-08c1-4c1e-bac1-08f597dd155e/volumes" Jul 09 15:28:59.046286 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:59.046196 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:28:59.046828 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:59.046440 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" containerID="cri-o://df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984" gracePeriod=30 Jul 09 15:28:59.338556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:59.338472 2558 patch_prober.go:28] interesting pod/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 15:28:59.338556 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:28:59.338539 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 15:29:04.294866 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.294839 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:29:04.406212 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406141 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406212 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406197 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406246 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406299 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406316 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406333 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406351 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406393 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406419 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc5rg\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg\") pod \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\" (UID: \"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4\") " Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406419 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406597 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406633 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:04.406660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406647 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.407021 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.406657 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:04.407628 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.407602 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:04.408784 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.408758 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:29:04.408922 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.408813 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:04.408986 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.408964 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg" (OuterVolumeSpecName: "kube-api-access-cc5rg") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "kube-api-access-cc5rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:29:04.409029 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.408968 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" (UID: "d225c2b2-b440-4bed-b6df-f4eaa80fb8f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:29:04.507107 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507077 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507107 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507104 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507107 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507113 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507121 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507131 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507139 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc5rg\" (UniqueName: \"kubernetes.io/projected/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-kube-api-access-cc5rg\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507147 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:04.507416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:04.507155 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:05.097644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.097607 2558 generic.go:356] "Generic (PLEG): container finished" podID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerID="df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984" exitCode=0 Jul 09 15:29:05.097850 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.097651 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" Jul 09 15:29:05.097850 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.097696 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerDied","Data":"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984"} Jul 09 15:29:05.097850 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.097762 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r" event={"ID":"d225c2b2-b440-4bed-b6df-f4eaa80fb8f4","Type":"ContainerDied","Data":"e6238905f51a0caff61024053f5738b2d20280b7428d926a723de09cd2090060"} Jul 09 15:29:05.097850 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.097792 2558 scope.go:117] "RemoveContainer" containerID="df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984" Jul 09 15:29:05.107353 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.107335 2558 scope.go:117] "RemoveContainer" containerID="b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f" Jul 09 15:29:05.115012 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.114994 2558 scope.go:117] "RemoveContainer" containerID="df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984" Jul 09 15:29:05.115270 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:29:05.115252 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984\": container with ID starting with df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984 not found: ID does not exist" containerID="df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984" Jul 09 15:29:05.115343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.115283 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984"} err="failed to get container status \"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984\": rpc error: code = NotFound desc = could not find container \"df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984\": container with ID starting with df050df4c864611c9e185261dcb9a98f250083b4049fa50c699a8c461e96f984 not found: ID does not exist" Jul 09 15:29:05.115343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.115307 2558 scope.go:117] "RemoveContainer" containerID="b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f" Jul 09 15:29:05.115563 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:29:05.115546 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f\": container with ID starting with b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f not found: ID does not exist" containerID="b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f" Jul 09 15:29:05.115602 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.115569 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f"} err="failed to get container status \"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f\": rpc error: code = NotFound desc = could not find container \"b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f\": container with ID starting with b33989c90855be099c827634a9034a0b71dc09ae872efcf66ef55e0d2399745f not found: ID does not exist" Jul 09 15:29:05.120959 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.120938 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:29:05.125256 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.125238 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-476308-openshift-default-55687f9977-rzd9r"] Jul 09 15:29:05.624112 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:05.624083 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" path="/var/lib/kubelet/pods/d225c2b2-b440-4bed-b6df-f4eaa80fb8f4/volumes" Jul 09 15:29:07.073764 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.073735 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074091 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074103 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074120 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074126 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074137 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91fa8976-08c1-4c1e-bac1-08f597dd155e" containerName="main" Jul 09 15:29:07.074167 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074142 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="91fa8976-08c1-4c1e-bac1-08f597dd155e" containerName="main" Jul 09 15:29:07.074427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074211 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="91fa8976-08c1-4c1e-bac1-08f597dd155e" containerName="main" Jul 09 15:29:07.074427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.074222 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:07.078519 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.078499 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.081464 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.081446 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-227dc3-openshift-default-dockercfg-khltq\"" Jul 09 15:29:07.090007 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.089989 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:07.228639 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228605 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.228831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228664 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.228831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.228831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228732 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bljrv\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.228831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228791 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.229077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228852 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.229077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.229077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.228958 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.229077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.229041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.329812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.329727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bljrv\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.329812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.329763 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.329812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.329785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.329812 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.329812 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330166 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.329971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330166 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330166 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330158 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330326 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330219 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330326 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330228 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330326 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330272 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330479 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330324 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330479 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330418 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330598 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330581 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.330930 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.330903 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.332284 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.332255 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.332284 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.332259 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.340532 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.340510 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.341185 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.341159 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bljrv\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv\") pod \"e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.388253 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.388224 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:07.537538 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.537503 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:07.540220 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:29:07.540188 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf845335b_f367_4521_a7b1_b0dd2b90cd04.slice/crio-4f599c23d863ba12aafae97695b9109424bdecc881535f88a99d585a62966c2b WatchSource:0}: Error finding container 4f599c23d863ba12aafae97695b9109424bdecc881535f88a99d585a62966c2b: Status 404 returned error can't find the container with id 4f599c23d863ba12aafae97695b9109424bdecc881535f88a99d585a62966c2b Jul 09 15:29:07.543343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.543166 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:07.543343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.543267 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:07.543343 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:07.543340 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:08.114660 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:08.114617 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerStarted","Data":"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9"} Jul 09 15:29:08.115077 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:08.114708 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerStarted","Data":"4f599c23d863ba12aafae97695b9109424bdecc881535f88a99d585a62966c2b"} Jul 09 15:29:08.136806 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:08.136749 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podStartSLOduration=1.136728844 podStartE2EDuration="1.136728844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:29:08.136345073 +0000 UTC m=+2080.999352243" watchObservedRunningTime="2026-07-09 15:29:08.136728844 +0000 UTC m=+2080.999736014" Jul 09 15:29:08.388937 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:08.388839 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:08.394693 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:08.394666 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:09.119817 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:09.119784 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:09.120821 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:09.120799 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:25.162016 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:25.161974 2558 patch_prober.go:28] interesting pod/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 15:29:25.162485 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:25.162050 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 15:29:27.217338 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.217303 2558 generic.go:356] "Generic (PLEG): container finished" podID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerID="5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9" exitCode=0 Jul 09 15:29:27.217751 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.217362 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerDied","Data":"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9"} Jul 09 15:29:27.217751 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.217740 2558 scope.go:117] "RemoveContainer" containerID="5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9" Jul 09 15:29:27.218731 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.218703 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:27.218785 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.218773 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:27.218821 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.218803 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:27.725694 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.725666 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:29:27.730426 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:27.730402 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjdnn_b15fa70c-505f-4d9d-8300-6c2991041fba/ovn-acl-logging/0.log" Jul 09 15:29:28.223023 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:28.222990 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerStarted","Data":"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c"} Jul 09 15:29:28.389230 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:28.389189 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:28.389472 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:28.389449 2558 patch_prober.go:28] interesting pod/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:29:28.389518 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:28.389498 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 09 15:29:29.389481 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:29.389443 2558 patch_prober.go:28] interesting pod/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:29:29.389920 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:29.389514 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 09 15:29:30.389373 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:30.389333 2558 patch_prober.go:28] interesting pod/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:29:30.389534 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:30.389404 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 09 15:29:31.389616 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:31.389580 2558 patch_prober.go:28] interesting pod/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 09 15:29:31.390028 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:31.389639 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 09 15:29:32.392378 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:32.392350 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:32.392738 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:32.392628 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:32.393371 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:32.393353 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:38.303708 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:38.303676 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:38.304246 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:38.303899 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" containerID="cri-o://e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c" gracePeriod=30 Jul 09 15:29:43.561695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.561667 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:43.659894 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659801 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.659894 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659841 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.659894 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659858 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.659894 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659892 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660229 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659929 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660229 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.659958 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660229 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660018 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bljrv\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660229 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660041 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660229 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660095 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert\") pod \"f845335b-f367-4521-a7b1-b0dd2b90cd04\" (UID: \"f845335b-f367-4521-a7b1-b0dd2b90cd04\") " Jul 09 15:29:43.660491 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660274 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:43.660491 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660383 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.660491 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660463 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:43.660667 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660562 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:29:43.660667 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660610 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:43.660962 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.660936 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data" (OuterVolumeSpecName: "istio-data") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:43.662149 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.662121 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:29:43.662263 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.662247 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv" (OuterVolumeSpecName: "kube-api-access-bljrv") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "kube-api-access-bljrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:29:43.662397 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.662378 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:29:43.662457 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.662432 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token" (OuterVolumeSpecName: "istio-token") pod "f845335b-f367-4521-a7b1-b0dd2b90cd04" (UID: "f845335b-f367-4521-a7b1-b0dd2b90cd04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:29:43.761659 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761624 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bljrv\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-kube-api-access-bljrv\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761659 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761651 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761659 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761661 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f845335b-f367-4521-a7b1-b0dd2b90cd04-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761659 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761670 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761958 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761677 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761958 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761687 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761958 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761694 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:43.761958 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:43.761703 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f845335b-f367-4521-a7b1-b0dd2b90cd04-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:29:44.299248 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.299218 2558 generic.go:356] "Generic (PLEG): container finished" podID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerID="e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c" exitCode=0 Jul 09 15:29:44.299459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.299285 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" Jul 09 15:29:44.299459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.299300 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerDied","Data":"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c"} Jul 09 15:29:44.299459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.299352 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c" event={"ID":"f845335b-f367-4521-a7b1-b0dd2b90cd04","Type":"ContainerDied","Data":"4f599c23d863ba12aafae97695b9109424bdecc881535f88a99d585a62966c2b"} Jul 09 15:29:44.299459 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.299372 2558 scope.go:117] "RemoveContainer" containerID="e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c" Jul 09 15:29:44.308697 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.308680 2558 scope.go:117] "RemoveContainer" containerID="5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9" Jul 09 15:29:44.316588 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.316564 2558 scope.go:117] "RemoveContainer" containerID="e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c" Jul 09 15:29:44.316828 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:29:44.316808 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c\": container with ID starting with e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c not found: ID does not exist" containerID="e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c" Jul 09 15:29:44.316891 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.316838 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c"} err="failed to get container status \"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c\": rpc error: code = NotFound desc = could not find container \"e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c\": container with ID starting with e025375c0b3565fb2f7fe6bb4be3abe4d8b4ae0bcfb0fbdc782dcc7b6c60ae8c not found: ID does not exist" Jul 09 15:29:44.316891 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.316861 2558 scope.go:117] "RemoveContainer" containerID="5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9" Jul 09 15:29:44.317116 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:29:44.317101 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9\": container with ID starting with 5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9 not found: ID does not exist" containerID="5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9" Jul 09 15:29:44.317154 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.317121 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9"} err="failed to get container status \"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9\": rpc error: code = NotFound desc = could not find container \"5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9\": container with ID starting with 5a33cbcbbb97a8f630725cab2b16c2d4eae62b584be4c1b2b8590419d014c5d9 not found: ID does not exist" Jul 09 15:29:44.324079 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.324056 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:44.328607 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:44.328589 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-227dc3-openshift-default-857f6b978f-xdc2c"] Jul 09 15:29:45.623596 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:45.623556 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" path="/var/lib/kubelet/pods/f845335b-f367-4521-a7b1-b0dd2b90cd04/volumes" Jul 09 15:29:46.602904 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.602861 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:29:46.603224 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.603212 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:29:46.603282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.603226 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:29:46.603319 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.603300 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:29:46.603319 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.603309 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d225c2b2-b440-4bed-b6df-f4eaa80fb8f4" containerName="istio-proxy" Jul 09 15:29:46.603319 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.603314 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:29:46.607678 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.607657 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.610836 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.610821 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-df6252-openshift-default-dockercfg-r98xr\"" Jul 09 15:29:46.621449 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.621426 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:29:46.687342 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687188 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687373 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687410 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687442 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687545 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687603 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687636 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.687834 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.687703 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2ht\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789059 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789022 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789059 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789060 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2ht\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789361 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789308 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789361 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789339 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789463 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789463 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789463 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789648 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789465 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789710 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789680 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789767 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789726 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.789820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.789764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.790125 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.790106 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.791974 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.791954 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.792141 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.792127 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.798885 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.798853 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2ht\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.798986 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.798936 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token\") pod \"e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:46.917298 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:46.917204 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:47.062784 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.062756 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:29:47.064712 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:29:47.064686 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95604458_93aa_4ab2_a668_ef9d09177e9b.slice/crio-24b026df8878d5fb3e6bd749b33c51df3b0919aeea2640ee9ee4c781ba84e0b6 WatchSource:0}: Error finding container 24b026df8878d5fb3e6bd749b33c51df3b0919aeea2640ee9ee4c781ba84e0b6: Status 404 returned error can't find the container with id 24b026df8878d5fb3e6bd749b33c51df3b0919aeea2640ee9ee4c781ba84e0b6 Jul 09 15:29:47.067195 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.067159 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:47.067282 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.067242 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:47.067328 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.067292 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:29:47.315664 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.315629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerStarted","Data":"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac"} Jul 09 15:29:47.315831 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.315685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerStarted","Data":"24b026df8878d5fb3e6bd749b33c51df3b0919aeea2640ee9ee4c781ba84e0b6"} Jul 09 15:29:47.339590 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.339541 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podStartSLOduration=1.339521283 podStartE2EDuration="1.339521283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:29:47.338422615 +0000 UTC m=+2120.201429808" watchObservedRunningTime="2026-07-09 15:29:47.339521283 +0000 UTC m=+2120.202528453" Jul 09 15:29:47.917912 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:47.917865 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:48.923170 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:48.923136 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:49.325405 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:49.325373 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:29:49.326253 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:29:49.326238 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:30:00.157818 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.157742 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:30:00.158208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.158122 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:30:00.158208 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.158133 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f845335b-f367-4521-a7b1-b0dd2b90cd04" containerName="istio-proxy" Jul 09 15:30:00.160957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.160940 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:30:00.165701 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.165682 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sszck\"" Jul 09 15:30:00.165833 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.165743 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 15:30:00.167039 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.167021 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 15:30:00.178718 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.178691 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:30:00.308518 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.308483 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcjpg\" (UniqueName: \"kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg\") pod \"maas-api-key-cleanup-29726850-m6h9b\" (UID: \"db06d32e-044a-47e4-9772-12d1956d8008\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:30:00.409632 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.409548 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcjpg\" (UniqueName: \"kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg\") pod \"maas-api-key-cleanup-29726850-m6h9b\" (UID: \"db06d32e-044a-47e4-9772-12d1956d8008\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:30:00.424487 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.424448 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcjpg\" (UniqueName: \"kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg\") pod \"maas-api-key-cleanup-29726850-m6h9b\" (UID: \"db06d32e-044a-47e4-9772-12d1956d8008\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:30:00.471477 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.471438 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:30:00.632238 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.632174 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:30:00.634530 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:30:00.634500 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb06d32e_044a_47e4_9772_12d1956d8008.slice/crio-afa2e1d5c4955257b03cc661015b533e80d1f72917b22483df76407ca56f3db3 WatchSource:0}: Error finding container afa2e1d5c4955257b03cc661015b533e80d1f72917b22483df76407ca56f3db3: Status 404 returned error can't find the container with id afa2e1d5c4955257b03cc661015b533e80d1f72917b22483df76407ca56f3db3 Jul 09 15:30:00.636783 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:00.636743 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:30:01.390855 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:01.390819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" event={"ID":"db06d32e-044a-47e4-9772-12d1956d8008","Type":"ContainerStarted","Data":"afa2e1d5c4955257b03cc661015b533e80d1f72917b22483df76407ca56f3db3"} Jul 09 15:30:03.401043 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.401009 2558 generic.go:356] "Generic (PLEG): container finished" podID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerID="a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac" exitCode=0 Jul 09 15:30:03.401464 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.401067 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerDied","Data":"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac"} Jul 09 15:30:03.401464 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.401430 2558 scope.go:117] "RemoveContainer" containerID="a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac" Jul 09 15:30:03.402398 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.402372 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:03.402466 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.402437 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:03.402506 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:03.402467 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 15:30:04.326293 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:04.326261 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:30:04.409278 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:04.409240 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerStarted","Data":"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6"} Jul 09 15:30:04.917652 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:04.917616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:30:04.918936 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:04.918907 2558 patch_prober.go:28] interesting pod/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:30:04.919088 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:04.918968 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 09 15:30:05.918052 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:05.918019 2558 patch_prober.go:28] interesting pod/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:30:05.918440 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:05.918075 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 09 15:30:06.420392 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:06.420360 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" event={"ID":"db06d32e-044a-47e4-9772-12d1956d8008","Type":"ContainerStarted","Data":"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d"} Jul 09 15:30:06.438399 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:06.438341 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" podStartSLOduration=1.011461879 podStartE2EDuration="6.438321718s" totalImagesPullingTime="5.426859839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:06.437424841 +0000 UTC m=+2139.300432012" watchObservedRunningTime="2026-07-09 15:30:06.438321718 +0000 UTC m=+2139.301328887" Jul 09 15:30:06.918601 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:06.918565 2558 patch_prober.go:28] interesting pod/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:30:06.918997 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:06.918621 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 09 15:30:07.918305 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:07.918267 2558 patch_prober.go:28] interesting pod/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 09 15:30:07.918492 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:07.918326 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 09 15:30:08.921926 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:08.921895 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:30:08.922388 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:08.922180 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:30:08.922866 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:30:08.922849 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:31:00.016555 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:00.016517 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:31:00.017125 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:00.016746 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" podUID="db06d32e-044a-47e4-9772-12d1956d8008" containerName="cleanup" containerID="cri-o://d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d" gracePeriod=30 Jul 09 15:31:30.199143 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.199119 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:31:30.276670 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.276603 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcjpg\" (UniqueName: \"kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg\") pod \"db06d32e-044a-47e4-9772-12d1956d8008\" (UID: \"db06d32e-044a-47e4-9772-12d1956d8008\") " Jul 09 15:31:30.278629 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.278600 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg" (OuterVolumeSpecName: "kube-api-access-lcjpg") pod "db06d32e-044a-47e4-9772-12d1956d8008" (UID: "db06d32e-044a-47e4-9772-12d1956d8008"). InnerVolumeSpecName "kube-api-access-lcjpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:31:30.377416 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.377392 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcjpg\" (UniqueName: \"kubernetes.io/projected/db06d32e-044a-47e4-9772-12d1956d8008-kube-api-access-lcjpg\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:31:30.813441 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.813406 2558 generic.go:356] "Generic (PLEG): container finished" podID="db06d32e-044a-47e4-9772-12d1956d8008" containerID="d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d" exitCode=137 Jul 09 15:31:30.813600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.813454 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" Jul 09 15:31:30.813600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.813461 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" event={"ID":"db06d32e-044a-47e4-9772-12d1956d8008","Type":"ContainerDied","Data":"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d"} Jul 09 15:31:30.813600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.813502 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b" event={"ID":"db06d32e-044a-47e4-9772-12d1956d8008","Type":"ContainerDied","Data":"afa2e1d5c4955257b03cc661015b533e80d1f72917b22483df76407ca56f3db3"} Jul 09 15:31:30.813600 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.813524 2558 scope.go:117] "RemoveContainer" containerID="d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d" Jul 09 15:31:30.821965 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.821949 2558 scope.go:117] "RemoveContainer" containerID="d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d" Jul 09 15:31:30.822220 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:31:30.822201 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d\": container with ID starting with d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d not found: ID does not exist" containerID="d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d" Jul 09 15:31:30.822266 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.822228 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d"} err="failed to get container status \"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d\": rpc error: code = NotFound desc = could not find container \"d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d\": container with ID starting with d945b41f70989776c2f454939e15255bd10448825e88c6547071587f377ccd6d not found: ID does not exist" Jul 09 15:31:30.838453 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.838432 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:31:30.849672 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:30.849651 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-m6h9b"] Jul 09 15:31:31.623803 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:31:31.623770 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db06d32e-044a-47e4-9772-12d1956d8008" path="/var/lib/kubelet/pods/db06d32e-044a-47e4-9772-12d1956d8008/volumes" Jul 09 15:33:14.555389 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:14.555355 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:33:14.555948 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:14.555587 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" containerID="cri-o://625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6" gracePeriod=30 Jul 09 15:33:19.819725 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.819643 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:33:19.909374 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909339 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909374 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909382 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909412 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909428 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909456 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909477 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909514 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909587 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.909644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909616 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m2ht\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht\") pod \"95604458-93aa-4ab2-a668-ef9d09177e9b\" (UID: \"95604458-93aa-4ab2-a668-ef9d09177e9b\") " Jul 09 15:33:19.910017 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909905 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:19.910017 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909924 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:19.910017 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.909939 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 15:33:19.910133 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.910000 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:19.910472 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.910443 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:19.911994 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.911963 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:33:19.912117 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.912043 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 15:33:19.912269 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.912253 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:33:19.912369 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:19.912351 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht" (OuterVolumeSpecName: "kube-api-access-9m2ht") pod "95604458-93aa-4ab2-a668-ef9d09177e9b" (UID: "95604458-93aa-4ab2-a668-ef9d09177e9b"). InnerVolumeSpecName "kube-api-access-9m2ht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010423 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-token\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010456 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95604458-93aa-4ab2-a668-ef9d09177e9b-istiod-ca-cert\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010464 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-certs\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010473 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-workload-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010481 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-podinfo\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010471 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010489 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-credential-socket\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010951 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010498 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-envoy\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010951 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010506 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m2ht\" (UniqueName: \"kubernetes.io/projected/95604458-93aa-4ab2-a668-ef9d09177e9b-kube-api-access-9m2ht\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.010951 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.010513 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95604458-93aa-4ab2-a668-ef9d09177e9b-istio-data\") on node \"ip-10-0-142-231.ec2.internal\" DevicePath \"\"" Jul 09 15:33:20.313978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.313938 2558 generic.go:356] "Generic (PLEG): container finished" podID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerID="625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6" exitCode=0 Jul 09 15:33:20.314248 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.313992 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" Jul 09 15:33:20.314248 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.314016 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerDied","Data":"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6"} Jul 09 15:33:20.314248 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.314067 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6" event={"ID":"95604458-93aa-4ab2-a668-ef9d09177e9b","Type":"ContainerDied","Data":"24b026df8878d5fb3e6bd749b33c51df3b0919aeea2640ee9ee4c781ba84e0b6"} Jul 09 15:33:20.314248 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.314090 2558 scope.go:117] "RemoveContainer" containerID="625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6" Jul 09 15:33:20.324303 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.324279 2558 scope.go:117] "RemoveContainer" containerID="a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac" Jul 09 15:33:20.334898 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.334834 2558 scope.go:117] "RemoveContainer" containerID="625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6" Jul 09 15:33:20.335298 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:33:20.335260 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6\": container with ID starting with 625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6 not found: ID does not exist" containerID="625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6" Jul 09 15:33:20.335415 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.335310 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6"} err="failed to get container status \"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6\": rpc error: code = NotFound desc = could not find container \"625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6\": container with ID starting with 625affd9f22d85368ffea5ae68b881a514eddedd8ec17ad445f2ebe80817cad6 not found: ID does not exist" Jul 09 15:33:20.335415 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.335333 2558 scope.go:117] "RemoveContainer" containerID="a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac" Jul 09 15:33:20.335671 ip-10-0-142-231 kubenswrapper[2558]: E0709 15:33:20.335648 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac\": container with ID starting with a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac not found: ID does not exist" containerID="a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac" Jul 09 15:33:20.335768 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.335678 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac"} err="failed to get container status \"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac\": rpc error: code = NotFound desc = could not find container \"a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac\": container with ID starting with a9bd3e453df79a4a199c13c4eed84ff6c93245d0574430acded537ac490490ac not found: ID does not exist" Jul 09 15:33:20.342795 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.342754 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:33:20.348497 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:20.348468 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-df6252-openshift-default-ddd9b4ccf-slsn6"] Jul 09 15:33:21.624389 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:21.624354 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" path="/var/lib/kubelet/pods/95604458-93aa-4ab2-a668-ef9d09177e9b/volumes" Jul 09 15:33:22.788007 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:22.787976 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55f4b7dddc-m8wgn_78c97e2c-e3ff-4369-a4ba-63bcd1c2d046/authorino/0.log" Jul 09 15:33:27.580548 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:27.580520 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dfzr7_a42114fe-c4a0-4d5a-8e1d-192845f031c8/manager/0.log" Jul 09 15:33:27.712411 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:27.712377 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6h24n_a7c673c8-b24c-4d4f-9ef7-1e91c41546af/manager/0.log" Jul 09 15:33:27.956491 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:27.956450 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lpc4j_4e609a77-2a0d-4292-b1cf-438eb31fc0d1/server/0.log" Jul 09 15:33:28.077511 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:28.077471 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rmn2x_838195b4-8bfc-485c-bb95-0b973e5e8864/manager/0.log" Jul 09 15:33:28.325796 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:28.325767 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5_d7f318c3-af8d-4f18-b1e6-c6fbebc86184/copy-manifests/0.log" Jul 09 15:33:28.334489 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:28.334459 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5_d7f318c3-af8d-4f18-b1e6-c6fbebc86184/manager/0.log" Jul 09 15:33:29.849947 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:29.849906 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55f4b7dddc-m8wgn_78c97e2c-e3ff-4369-a4ba-63bcd1c2d046/authorino/0.log" Jul 09 15:33:29.970394 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:29.970360 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ngwg6_1fe21115-fa3a-4985-8da5-385d942ebc84/manager/0.log" Jul 09 15:33:30.090126 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:30.090070 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h59kx_4b0659de-50e1-4250-b551-0b1b38aeab56/manager/0.log" Jul 09 15:33:30.216054 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:30.216027 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qt8b5_66b4526c-8aa9-4466-884f-6c091b4e9c6a/kuadrant-console-plugin/0.log" Jul 09 15:33:30.355467 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:30.355427 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7j9pg_39e1dc0b-d5aa-469d-8c83-aa3f5aca7133/registry-server/0.log" Jul 09 15:33:30.496832 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:30.496742 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lbq2v_7cb8cc90-5019-4bf0-b85b-37baf37d0b44/manager/0.log" Jul 09 15:33:30.765251 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:30.765163 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6vvcd_c9040363-d694-4579-8a43-306e99815a98/manager/0.log" Jul 09 15:33:31.137150 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:31.137057 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mt8xg_f8e766ed-bc9a-494f-af79-b469aa2ad1a4/istio-proxy/0.log" Jul 09 15:33:31.495156 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:31.495127 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99cb46dbc-m4tlk_514dc168-0970-43d3-a336-91e850eb700c/kube-auth-proxy/0.log" Jul 09 15:33:31.763772 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:31.763684 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qdsb8_d5768f7e-1987-40ce-a9cc-7f63bcdf1dde/payload-pre-processing/0.log" Jul 09 15:33:32.650022 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:32.649992 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xmtxs_4e78d009-16d2-4c08-917e-9d99b0e16066/main/0.log" Jul 09 15:33:32.778038 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:32.777998 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxgzl7_2a0badc3-ba07-41cc-9b74-571d75bec163/main/0.log" Jul 09 15:33:37.827942 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.827905 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8k5dp/must-gather-tzqtg"] Jul 09 15:33:37.828371 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828352 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.828486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828376 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.828486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828399 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.828486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828406 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.828486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828412 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db06d32e-044a-47e4-9772-12d1956d8008" containerName="cleanup" Jul 09 15:33:37.828486 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828417 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="db06d32e-044a-47e4-9772-12d1956d8008" containerName="cleanup" Jul 09 15:33:37.828655 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828489 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="db06d32e-044a-47e4-9772-12d1956d8008" containerName="cleanup" Jul 09 15:33:37.828655 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828500 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.828655 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.828507 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="95604458-93aa-4ab2-a668-ef9d09177e9b" containerName="istio-proxy" Jul 09 15:33:37.831663 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.831647 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.834827 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.834804 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8k5dp\"/\"openshift-service-ca.crt\"" Jul 09 15:33:37.834979 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.834804 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8k5dp\"/\"default-dockercfg-ftn9x\"" Jul 09 15:33:37.835196 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.835183 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8k5dp\"/\"kube-root-ca.crt\"" Jul 09 15:33:37.845955 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.845928 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8k5dp/must-gather-tzqtg"] Jul 09 15:33:37.865560 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.865526 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/76b175a3-95c3-46dc-9c4e-301304d27195-must-gather-output\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.865560 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.865567 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9gz\" (UniqueName: \"kubernetes.io/projected/76b175a3-95c3-46dc-9c4e-301304d27195-kube-api-access-mb9gz\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.966440 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.966402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/76b175a3-95c3-46dc-9c4e-301304d27195-must-gather-output\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.966644 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.966463 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9gz\" (UniqueName: \"kubernetes.io/projected/76b175a3-95c3-46dc-9c4e-301304d27195-kube-api-access-mb9gz\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.966762 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.966743 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/76b175a3-95c3-46dc-9c4e-301304d27195-must-gather-output\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:37.977153 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:37.977125 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9gz\" (UniqueName: \"kubernetes.io/projected/76b175a3-95c3-46dc-9c4e-301304d27195-kube-api-access-mb9gz\") pod \"must-gather-tzqtg\" (UID: \"76b175a3-95c3-46dc-9c4e-301304d27195\") " pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:38.140927 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:38.140805 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" Jul 09 15:33:38.287838 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:38.287769 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8k5dp/must-gather-tzqtg"] Jul 09 15:33:38.292970 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:33:38.292841 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b175a3_95c3_46dc_9c4e_301304d27195.slice/crio-9e6b47a13ac958c07a278bc372d0fd2e7ff7a72d7bce57d71ff7c21bc3f82525 WatchSource:0}: Error finding container 9e6b47a13ac958c07a278bc372d0fd2e7ff7a72d7bce57d71ff7c21bc3f82525: Status 404 returned error can't find the container with id 9e6b47a13ac958c07a278bc372d0fd2e7ff7a72d7bce57d71ff7c21bc3f82525 Jul 09 15:33:38.401565 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:38.401467 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" event={"ID":"76b175a3-95c3-46dc-9c4e-301304d27195","Type":"ContainerStarted","Data":"9e6b47a13ac958c07a278bc372d0fd2e7ff7a72d7bce57d71ff7c21bc3f82525"} Jul 09 15:33:39.410561 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:39.410523 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" event={"ID":"76b175a3-95c3-46dc-9c4e-301304d27195","Type":"ContainerStarted","Data":"2340c02b70ac191428e683ebe46e7f43a620135c4d2bb68a6b9736b8ea711fae"} Jul 09 15:33:39.410988 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:39.410579 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" event={"ID":"76b175a3-95c3-46dc-9c4e-301304d27195","Type":"ContainerStarted","Data":"36f401d47a8788e98b48444f669ad1685c4b9ebd7c7c90895710e6ebd0666afa"} Jul 09 15:33:39.435948 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:39.435821 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8k5dp/must-gather-tzqtg" podStartSLOduration=1.557855024 podStartE2EDuration="2.435803008s" totalImagesPullingTime="877.947984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:39.435202282 +0000 UTC m=+2352.298209451" watchObservedRunningTime="2026-07-09 15:33:39.435803008 +0000 UTC m=+2352.298810200" Jul 09 15:33:41.179081 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:41.179043 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jlq7p_ce2db69a-8723-42cc-8359-292bf2d8e76d/global-pull-secret-syncer/0.log" Jul 09 15:33:41.411789 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:41.411752 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d88j8_488409ee-00bf-48aa-bb9a-d16af490d712/konnectivity-agent/0.log" Jul 09 15:33:41.568947 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:41.568914 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-231.ec2.internal_dfc52fc8e6e2f32cc911f6a116c3e1f0/haproxy/0.log" Jul 09 15:33:46.394898 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.394681 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55f4b7dddc-m8wgn_78c97e2c-e3ff-4369-a4ba-63bcd1c2d046/authorino/0.log" Jul 09 15:33:46.435820 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.435781 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ngwg6_1fe21115-fa3a-4985-8da5-385d942ebc84/manager/0.log" Jul 09 15:33:46.486920 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.486851 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h59kx_4b0659de-50e1-4250-b551-0b1b38aeab56/manager/0.log" Jul 09 15:33:46.530692 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.530663 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qt8b5_66b4526c-8aa9-4466-884f-6c091b4e9c6a/kuadrant-console-plugin/0.log" Jul 09 15:33:46.590216 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.590178 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7j9pg_39e1dc0b-d5aa-469d-8c83-aa3f5aca7133/registry-server/0.log" Jul 09 15:33:46.795838 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.792469 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-lbq2v_7cb8cc90-5019-4bf0-b85b-37baf37d0b44/manager/0.log" Jul 09 15:33:46.958665 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:46.958622 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6vvcd_c9040363-d694-4579-8a43-306e99815a98/manager/0.log" Jul 09 15:33:49.392264 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:49.392224 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrf2b_275ac97f-071b-4d99-b75a-5df17b478ae5/node-exporter/0.log" Jul 09 15:33:49.429843 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:49.429807 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrf2b_275ac97f-071b-4d99-b75a-5df17b478ae5/kube-rbac-proxy/0.log" Jul 09 15:33:49.514549 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:49.514518 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zrf2b_275ac97f-071b-4d99-b75a-5df17b478ae5/init-textfile/0.log" Jul 09 15:33:51.517021 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.516984 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-hqmnq_a8677844-41dc-4c2c-b82a-6ecf37554f9d/networking-console-plugin/0.log" Jul 09 15:33:51.702310 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.702276 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh"] Jul 09 15:33:51.708357 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.708329 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.718695 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.718670 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh"] Jul 09 15:33:51.828687 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.828590 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-podres\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.828687 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.828647 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-proc\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.828957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.828703 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-lib-modules\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.828957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.828740 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdv5t\" (UniqueName: \"kubernetes.io/projected/028a9e07-593c-4540-8284-e742b9665e1c-kube-api-access-wdv5t\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.828957 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.828784 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-sys\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930120 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930084 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-podres\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930120 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930124 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-proc\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-lib-modules\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930257 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdv5t\" (UniqueName: \"kubernetes.io/projected/028a9e07-593c-4540-8284-e742b9665e1c-kube-api-access-wdv5t\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930272 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-podres\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930280 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-proc\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-sys\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930346 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930324 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-sys\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.930640 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.930357 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/028a9e07-593c-4540-8284-e742b9665e1c-lib-modules\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:51.940669 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:51.940632 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdv5t\" (UniqueName: \"kubernetes.io/projected/028a9e07-593c-4540-8284-e742b9665e1c-kube-api-access-wdv5t\") pod \"perf-node-gather-daemonset-wmcwh\" (UID: \"028a9e07-593c-4540-8284-e742b9665e1c\") " pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:52.021501 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.021468 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:52.199944 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.199921 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh"] Jul 09 15:33:52.201963 ip-10-0-142-231 kubenswrapper[2558]: W0709 15:33:52.201929 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod028a9e07_593c_4540_8284_e742b9665e1c.slice/crio-725f6f320a4d69ef89d82b20030a6378882f0c6cb94c8c9957fef84585509e3e WatchSource:0}: Error finding container 725f6f320a4d69ef89d82b20030a6378882f0c6cb94c8c9957fef84585509e3e: Status 404 returned error can't find the container with id 725f6f320a4d69ef89d82b20030a6378882f0c6cb94c8c9957fef84585509e3e Jul 09 15:33:52.513243 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.513207 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" event={"ID":"028a9e07-593c-4540-8284-e742b9665e1c","Type":"ContainerStarted","Data":"8d5f16ba769ab245a84572dbd33a42ba4cee0923982fa3a0876ec4692bd9f63a"} Jul 09 15:33:52.513427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.513263 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" event={"ID":"028a9e07-593c-4540-8284-e742b9665e1c","Type":"ContainerStarted","Data":"725f6f320a4d69ef89d82b20030a6378882f0c6cb94c8c9957fef84585509e3e"} Jul 09 15:33:52.513427 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.513372 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:33:52.533614 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:52.533563 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" podStartSLOduration=1.533546015 podStartE2EDuration="1.533546015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:52.532572251 +0000 UTC m=+2365.395579419" watchObservedRunningTime="2026-07-09 15:33:52.533546015 +0000 UTC m=+2365.396553215" Jul 09 15:33:54.566333 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:54.566293 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz5xw_1078985e-6c76-4495-b4f0-43cd7eeb645b/dns/0.log" Jul 09 15:33:54.593782 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:54.593755 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zz5xw_1078985e-6c76-4495-b4f0-43cd7eeb645b/kube-rbac-proxy/0.log" Jul 09 15:33:54.702836 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:54.702811 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4lnm_f1a9f981-930d-4fbd-a6a7-43e8bf453208/dns-node-resolver/0.log" Jul 09 15:33:55.346094 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:55.346059 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-glqpm_7e6cb45c-1646-42d3-b22f-89038504faf9/node-ca/0.log" Jul 09 15:33:56.393636 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:56.393591 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mt8xg_f8e766ed-bc9a-494f-af79-b469aa2ad1a4/istio-proxy/0.log" Jul 09 15:33:56.633995 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:56.633964 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99cb46dbc-m4tlk_514dc168-0970-43d3-a336-91e850eb700c/kube-auth-proxy/0.log" Jul 09 15:33:56.771004 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:56.770971 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qdsb8_d5768f7e-1987-40ce-a9cc-7f63bcdf1dde/payload-pre-processing/0.log" Jul 09 15:33:57.625842 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:57.625812 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dgnf8_2010273d-2add-4376-9b00-452a615ad1ce/serve-healthcheck-canary/0.log" Jul 09 15:33:58.196779 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:58.196738 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kgdlk_5ed4f169-1b94-4415-a199-420f3d2858cd/kube-rbac-proxy/0.log" Jul 09 15:33:58.223978 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:58.223950 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kgdlk_5ed4f169-1b94-4415-a199-420f3d2858cd/exporter/0.log" Jul 09 15:33:58.251424 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:58.251389 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kgdlk_5ed4f169-1b94-4415-a199-420f3d2858cd/extractor/0.log" Jul 09 15:33:58.533558 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:33:58.533518 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8k5dp/perf-node-gather-daemonset-wmcwh" Jul 09 15:34:00.992939 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:00.992826 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-dfzr7_a42114fe-c4a0-4d5a-8e1d-192845f031c8/manager/0.log" Jul 09 15:34:01.174235 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:01.174197 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6h24n_a7c673c8-b24c-4d4f-9ef7-1e91c41546af/manager/0.log" Jul 09 15:34:01.330704 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:01.330669 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lpc4j_4e609a77-2a0d-4292-b1cf-438eb31fc0d1/server/0.log" Jul 09 15:34:01.429720 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:01.429692 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rmn2x_838195b4-8bfc-485c-bb95-0b973e5e8864/manager/0.log" Jul 09 15:34:01.526983 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:01.526949 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5_d7f318c3-af8d-4f18-b1e6-c6fbebc86184/manager/0.log" Jul 09 15:34:01.563577 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:01.563546 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fc7dfbfc4-rt4d5_d7f318c3-af8d-4f18-b1e6-c6fbebc86184/copy-manifests/0.log" Jul 09 15:34:02.670558 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:02.670519 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mt8xg_f8e766ed-bc9a-494f-af79-b469aa2ad1a4/istio-proxy/0.log" Jul 09 15:34:02.881961 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:02.881927 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99cb46dbc-m4tlk_514dc168-0970-43d3-a336-91e850eb700c/kube-auth-proxy/0.log" Jul 09 15:34:02.988659 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:02.988628 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qdsb8_d5768f7e-1987-40ce-a9cc-7f63bcdf1dde/payload-pre-processing/0.log" Jul 09 15:34:04.472255 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:04.472219 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mt8xg_f8e766ed-bc9a-494f-af79-b469aa2ad1a4/istio-proxy/0.log" Jul 09 15:34:04.692414 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:04.692355 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-99cb46dbc-m4tlk_514dc168-0970-43d3-a336-91e850eb700c/kube-auth-proxy/0.log" Jul 09 15:34:04.797501 ip-10-0-142-231 kubenswrapper[2558]: I0709 15:34:04.797398 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qdsb8_d5768f7e-1987-40ce-a9cc-7f63bcdf1dde/payload-pre-processing/0.log"