Jul 08 11:13:02.324233 ip-10-0-140-148 systemd[1]: Starting Kubernetes Kubelet... Jul 08 11:13:02.757334 ip-10-0-140-148 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 11:13:02.757334 ip-10-0-140-148 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 11:13:02.757334 ip-10-0-140-148 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 11:13:02.757334 ip-10-0-140-148 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 11:13:02.757334 ip-10-0-140-148 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 11:13:02.760055 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.759924 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 11:13:02.764973 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764959 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.764973 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764973 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764976 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764979 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764982 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764985 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764989 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764992 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764995 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.764997 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765001 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765003 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765006 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765009 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765011 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765014 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765016 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765019 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765021 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765024 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.765038 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765026 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765029 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765031 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765034 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765037 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765040 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765042 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765045 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765049 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765054 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765056 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765059 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765061 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765064 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765066 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765069 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765071 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765073 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765076 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.765485 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765079 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765082 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765084 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765086 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765089 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765091 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765093 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765096 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765098 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765101 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765103 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765105 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765108 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765112 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765115 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765119 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765122 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765125 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765128 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765130 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.765940 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765133 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765136 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765138 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765141 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765143 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765146 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765149 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765153 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765156 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765160 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765162 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765165 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765168 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765170 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765173 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765176 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765178 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765180 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765183 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.766440 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765185 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765187 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765190 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765192 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765194 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765197 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765199 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765203 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765206 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765208 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765211 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765213 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765216 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765218 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765221 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765223 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765226 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765228 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765231 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765233 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.766901 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765236 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765937 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765958 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765961 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765966 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765970 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765973 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765976 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765979 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765982 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765984 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765987 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765989 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765992 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765994 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765997 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.765999 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766002 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766004 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.767392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766009 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766014 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766017 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766020 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766023 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766026 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766028 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766030 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766033 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766035 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766038 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766040 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766042 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766045 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766047 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766049 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766051 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766054 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766056 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766059 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.767856 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766061 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766064 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766066 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766069 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766071 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766074 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766076 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766078 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766080 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766083 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766085 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766087 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766090 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766093 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766096 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766098 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766101 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766103 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766106 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766108 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.768360 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766111 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766113 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766116 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766118 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766120 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766123 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766128 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766131 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766134 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766137 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766139 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766142 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766144 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766150 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766152 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766155 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766157 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766159 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766162 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.768852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766164 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766167 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766169 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766171 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766174 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766176 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766179 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766182 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766185 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766187 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766190 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766192 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766195 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766197 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766199 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766201 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766204 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766206 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766209 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766211 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.769323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766213 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766216 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766583 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766587 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766590 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766593 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766596 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766599 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766601 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766604 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766606 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766608 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766611 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766613 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766616 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766618 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766621 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766625 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766628 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766631 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.769825 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766633 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766636 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766638 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766643 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766646 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766649 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766652 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766655 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766658 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766660 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766663 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766665 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766667 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766670 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766673 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766675 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766678 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766680 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766683 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.770313 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766686 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766688 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766691 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766693 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766696 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766698 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766700 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766703 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766705 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766708 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766712 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766715 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766718 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766721 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766723 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766726 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766729 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766733 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766736 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766739 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.770756 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766741 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766745 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766749 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766752 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766755 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766758 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766760 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766763 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766766 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766769 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766771 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766774 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766776 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766779 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766782 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766784 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766787 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766789 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766791 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.771284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766794 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766796 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766799 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766802 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766807 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766809 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766812 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766814 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766817 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766819 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766822 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766824 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766828 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766830 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766833 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766835 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766838 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766840 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766843 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766845 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.771739 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766847 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766850 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.766852 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766924 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766930 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766936 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766940 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766958 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766961 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766966 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766970 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766973 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766975 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766978 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766981 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766986 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766989 2559 flags.go:64] FLAG: --cgroup-root="" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766992 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766995 2559 flags.go:64] FLAG: --client-ca-file="" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.766998 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767001 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767008 2559 flags.go:64] FLAG: --cluster-domain="" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767011 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767014 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 11:13:02.772229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767017 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767020 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767023 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767027 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767030 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767033 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767036 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767039 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767042 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767045 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767048 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767052 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767055 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767057 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767060 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767063 2559 flags.go:64] FLAG: --enable-server="true" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767066 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767070 2559 flags.go:64] FLAG: --event-burst="100" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767073 2559 flags.go:64] FLAG: --event-qps="50" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767076 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767079 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767081 2559 flags.go:64] FLAG: --eviction-hard="" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767085 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767090 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767095 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 11:13:02.772976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767098 2559 flags.go:64] FLAG: --eviction-soft="" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767100 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767103 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767106 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767109 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767112 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767114 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767117 2559 flags.go:64] FLAG: --feature-gates="" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767121 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767124 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767127 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767130 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767132 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767135 2559 flags.go:64] FLAG: --help="false" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767138 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-140-148.ec2.internal" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767141 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767144 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767146 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767150 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767153 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767155 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767158 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767161 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 11:13:02.773565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767164 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767167 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767195 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767218 2559 flags.go:64] FLAG: --kube-reserved="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767224 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767228 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767232 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767235 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767239 2559 flags.go:64] FLAG: --lock-file="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767242 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767246 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767250 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767256 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767259 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767262 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767265 2559 flags.go:64] FLAG: --logging-format="text" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767268 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767272 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767275 2559 flags.go:64] FLAG: --manifest-url="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767278 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767284 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767287 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767291 2559 flags.go:64] FLAG: --max-pods="110" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767295 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767298 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 11:13:02.774167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767301 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767303 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767306 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767309 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767312 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767321 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767324 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767327 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767329 2559 flags.go:64] FLAG: --pod-cidr="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767332 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767341 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767344 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767347 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767350 2559 flags.go:64] FLAG: --port="10250" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767353 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767356 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fd306679dcbcdf17" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767359 2559 flags.go:64] FLAG: --qos-reserved="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767362 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767365 2559 flags.go:64] FLAG: --register-node="true" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767367 2559 flags.go:64] FLAG: --register-with-taints="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767372 2559 flags.go:64] FLAG: --registry-burst="10" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767374 2559 flags.go:64] FLAG: --registry-qps="5" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767377 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767380 2559 flags.go:64] FLAG: --reserved-memory="" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767383 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 11:13:02.775153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767387 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767390 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767392 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767395 2559 flags.go:64] FLAG: --runonce="false" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767398 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767401 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767404 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767407 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767409 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767412 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767415 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767418 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767421 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767424 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767426 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767429 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767432 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767435 2559 flags.go:64] FLAG: --system-cgroups="" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767439 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767445 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767448 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767451 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767456 2559 flags.go:64] FLAG: --tls-min-version="" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767459 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767462 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 11:13:02.776207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767466 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767468 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767471 2559 flags.go:64] FLAG: --v="2" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767475 2559 flags.go:64] FLAG: --version="false" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767479 2559 flags.go:64] FLAG: --vmodule="" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767482 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.767486 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767615 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767619 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767624 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767628 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767631 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767634 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767637 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767639 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767642 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767645 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767648 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767650 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767652 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767655 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.777335 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767657 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767660 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767662 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767665 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767670 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767673 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767676 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767679 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767681 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767684 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767686 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767689 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767691 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767694 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767696 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767699 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767701 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767703 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767706 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767708 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.778193 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767711 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767714 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767716 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767719 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767721 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767724 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767726 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767729 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767731 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767734 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767736 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767738 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767741 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767743 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767746 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767748 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767752 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767755 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767758 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767760 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.778678 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767763 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767765 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767767 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767770 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767772 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767775 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767777 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767779 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767782 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767784 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767786 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767789 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767791 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767794 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767796 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767798 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767801 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767803 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767806 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.779504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767809 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767811 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767814 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767816 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767819 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767821 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767825 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767829 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767832 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767836 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767839 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767842 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767845 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767848 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767850 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767853 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767856 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767858 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767861 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767864 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.780288 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767866 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767869 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767871 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767874 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767876 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.767880 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.768560 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.776538 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.776557 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.776582 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.776591 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776674 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776682 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776688 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.780790 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776693 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776698 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776702 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776707 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776712 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776716 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776720 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776725 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776729 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776733 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776737 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776741 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776748 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776754 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776760 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776765 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776770 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776775 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776779 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.781294 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776783 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776787 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776791 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776796 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776800 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776804 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776809 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776814 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776819 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776823 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776827 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776831 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776835 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776839 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776844 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776848 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776852 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776856 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776860 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.781766 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776864 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776868 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776872 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776876 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776881 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776885 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776889 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776893 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776897 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776904 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776909 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776914 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776918 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776923 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776927 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776931 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776935 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776939 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776962 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776967 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.782272 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776972 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776976 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776981 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776986 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776990 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776994 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.776997 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777001 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777006 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777010 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777014 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777018 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777022 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777026 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777030 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777034 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777039 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777043 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777046 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.782765 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777051 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777057 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777063 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777067 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777071 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777075 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777080 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777084 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777089 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777093 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777097 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777101 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777105 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777109 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777115 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777119 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777123 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777127 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.783351 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777131 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.777139 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777320 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777330 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777334 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777338 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777343 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777347 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777354 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777359 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777364 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777368 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777373 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777378 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777382 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 11:13:02.783798 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777387 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777392 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777396 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777400 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777404 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777409 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777413 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777418 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777422 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777426 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777430 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777435 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777439 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777443 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777449 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777453 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777457 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777463 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777468 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 11:13:02.784208 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777473 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777477 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777481 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777485 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777489 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777494 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777498 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777502 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777506 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777510 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777514 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777518 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777522 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777526 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777530 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777534 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777538 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777543 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777547 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777551 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 11:13:02.784675 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777555 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777559 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777563 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777567 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777571 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777575 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777579 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777583 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777588 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777592 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777598 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777604 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777608 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777613 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777617 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777621 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777625 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777629 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 11:13:02.785197 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777633 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777637 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777641 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777645 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777649 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777653 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777657 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777661 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777665 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777669 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777673 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777677 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777681 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777685 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777689 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777693 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777697 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777701 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777705 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777708 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 11:13:02.785632 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777712 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777716 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777721 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777725 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777729 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777733 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777737 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777741 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:02.777744 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.777752 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.777895 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.781064 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.782873 2559 server.go:1019] "Starting client certificate rotation" Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.782986 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 11:13:02.786150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.783049 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 11:13:02.786511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.785795 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 11:13:02.793237 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.793212 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 11:13:02.807302 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.807285 2559 log.go:25] "Validated CRI v1 runtime API" Jul 08 11:13:02.809707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.809685 2559 log.go:25] "Validated CRI v1 image API" Jul 08 11:13:02.810525 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.810508 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 11:13:02.814181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.814150 2559 fs.go:135] Filesystem UUIDs: map[1b36e2d6-7eec-46d4-b2ca-05a8a5769788:/dev/nvme0n1p3 5bedd959-0449-4205-a071-a39d5c5f4c65:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 08 11:13:02.814276 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.814180 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 11:13:02.820104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.819972 2559 manager.go:216] Machine: {Timestamp:2026-07-08 11:13:02.818632293 +0000 UTC m=+0.379966199 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3056675 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21da969a80952276c17bb5e83d4356 SystemUUID:ec21da96-9a80-9522-76c1-7bb5e83d4356 BootID:68e2af12-9626-4f59-9dfa-0dfa84988cfe Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:24:8b:28:a6:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:24:8b:28:a6:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:8d:14:02:41:87 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 11:13:02.820104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.820094 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 11:13:02.820279 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.820208 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 11:13:02.821382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.821354 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 11:13:02.821553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.821386 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 11:13:02.822294 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.822281 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 11:13:02.822355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.822299 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 11:13:02.822355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.822310 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 11:13:02.823182 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.823170 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 11:13:02.823367 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.823352 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 11:13:02.825156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.825142 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 11:13:02.829965 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.829936 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 08 11:13:02.830101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830090 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 11:13:02.830156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830130 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 08 11:13:02.830156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830145 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 11:13:02.830243 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830164 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 11:13:02.830243 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830178 2559 kubelet.go:398] "Adding apiserver pod source" Jul 08 11:13:02.830243 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830192 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 11:13:02.830582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830570 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 11:13:02.830698 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.830595 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 11:13:02.833896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.833333 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 11:13:02.835839 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.835822 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 11:13:02.835897 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.835854 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 11:13:02.837260 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837247 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837263 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837269 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837274 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837282 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837288 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837295 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837301 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837307 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837312 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837321 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 11:13:02.837331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.837326 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 11:13:02.838192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.838181 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 11:13:02.838192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.838193 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 11:13:02.841481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.841465 2559 server.go:1268] "Started kubelet" Jul 08 11:13:02.841569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.841531 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 11:13:02.841637 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.841577 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 11:13:02.841684 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.841667 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 11:13:02.846326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.846296 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 11:13:02.846882 ip-10-0-140-148 systemd[1]: Started Kubernetes Kubelet. Jul 08 11:13:02.847117 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.847040 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 11:13:02.847297 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.847269 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 11:13:02.847531 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.847488 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 11:13:02.848315 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.848296 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 08 11:13:02.848557 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.848525 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 11:13:02.852611 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.851483 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-148.ec2.internal.18c04c2700e93af0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-148.ec2.internal,UID:ip-10-0-140-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-148.ec2.internal,},FirstTimestamp:2026-07-08 11:13:02.841436912 +0000 UTC m=+0.402770818,LastTimestamp:2026-07-08 11:13:02.841436912 +0000 UTC m=+0.402770818,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-148.ec2.internal,}" Jul 08 11:13:02.856862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.856845 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 11:13:02.856923 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.856869 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 11:13:02.857013 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.856998 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:02.857659 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.857630 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 11:13:02.857659 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.857652 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 11:13:02.857796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.857635 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 11:13:02.857796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.857746 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 08 11:13:02.857796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.857753 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 08 11:13:02.859162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859143 2559 factory.go:55] Registering systemd factory Jul 08 11:13:02.859162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859160 2559 factory.go:223] Registration of the systemd container factory successfully Jul 08 11:13:02.859394 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859379 2559 factory.go:153] Registering CRI-O factory Jul 08 11:13:02.859394 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859393 2559 factory.go:223] Registration of the crio container factory successfully Jul 08 11:13:02.859489 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859438 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 11:13:02.859489 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859458 2559 factory.go:103] Registering Raw factory Jul 08 11:13:02.859489 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859468 2559 manager.go:1196] Started watching for new ooms in manager Jul 08 11:13:02.859799 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.859786 2559 manager.go:349] Starting recovery of all containers Jul 08 11:13:02.869516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.869499 2559 manager.go:354] Recovery completed Jul 08 11:13:02.871741 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.871710 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 11:13:02.871962 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.871924 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 11:13:02.873922 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.873910 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:02.875728 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.875705 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:02.875803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.875740 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:02.875803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.875751 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:02.876415 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.876400 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 11:13:02.876415 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.876415 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 11:13:02.876537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.876416 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:02.876537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.876434 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 08 11:13:02.877048 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.877034 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:02.877102 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.877062 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:02.877102 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.877072 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:02.878831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.878816 2559 policy_none.go:49] "None policy: Start" Jul 08 11:13:02.878831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.878831 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 11:13:02.878989 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.878841 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 11:13:02.880894 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.880880 2559 policy_none.go:47] "Start" Jul 08 11:13:02.889493 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.889428 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-148.ec2.internal.18c04c2702f47648 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-148.ec2.internal,UID:ip-10-0-140-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-148.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-148.ec2.internal,},FirstTimestamp:2026-07-08 11:13:02.875727432 +0000 UTC m=+0.437061338,LastTimestamp:2026-07-08 11:13:02.875727432 +0000 UTC m=+0.437061338,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-148.ec2.internal,}" Jul 08 11:13:02.895296 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.895255 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cd4bw" Jul 08 11:13:02.899473 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.899396 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-148.ec2.internal.18c04c2702f4bc29 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-148.ec2.internal,UID:ip-10-0-140-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-148.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-148.ec2.internal,},FirstTimestamp:2026-07-08 11:13:02.875745321 +0000 UTC m=+0.437079227,LastTimestamp:2026-07-08 11:13:02.875745321 +0000 UTC m=+0.437079227,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-148.ec2.internal,}" Jul 08 11:13:02.902078 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.902058 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cd4bw" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922507 2559 manager.go:337] "Starting Device Plugin manager" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.922536 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922549 2559 server.go:87] "Starting device plugin registration server" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922808 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922818 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922898 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922989 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.922997 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.923539 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 11:13:02.933758 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.923584 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:02.981551 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.981510 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 11:13:02.982881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.982854 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 11:13:02.983015 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.982892 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 11:13:02.983015 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.982915 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 11:13:02.983015 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:02.982970 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 11:13:02.986288 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:02.986266 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 11:13:03.023466 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.023382 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.024572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.024556 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.024658 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.024591 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.024658 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.024605 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.024658 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.024627 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.037886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.037864 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.038020 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.037891 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-148.ec2.internal\": node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.077750 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.077722 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.083058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.083027 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal"] Jul 08 11:13:03.083117 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.083099 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.083963 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.083932 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.084058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.083985 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.084058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.084000 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.086393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.086378 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.086497 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.086484 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.086546 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.086523 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.087131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087103 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.087131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087129 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.087255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087138 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.087255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087103 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.087255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087198 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.087255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.087207 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.089383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.089368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.089454 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.089392 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.090050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.090022 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.090050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.090048 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.090188 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.090057 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.121689 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.121665 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-148.ec2.internal\" not found" node="ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.126126 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.126111 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-148.ec2.internal\" not found" node="ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.158498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.158469 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.158614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.158510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.158614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.158527 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2339c7fd5cf13a79b0441cd2cf2b5de7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-148.ec2.internal\" (UID: \"2339c7fd5cf13a79b0441cd2cf2b5de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.178005 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.177980 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.258856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.258856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.259098 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2339c7fd5cf13a79b0441cd2cf2b5de7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-148.ec2.internal\" (UID: \"2339c7fd5cf13a79b0441cd2cf2b5de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.259098 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258934 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.259098 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258932 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/97e14242d5a8b7e59aec414a566b1b75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal\" (UID: \"97e14242d5a8b7e59aec414a566b1b75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.259098 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.258936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2339c7fd5cf13a79b0441cd2cf2b5de7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-148.ec2.internal\" (UID: \"2339c7fd5cf13a79b0441cd2cf2b5de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.278970 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.278931 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.379700 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.379673 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.423864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.423832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.428856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.428714 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:03.480015 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.479982 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.580591 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.580520 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.681025 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.680985 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.781614 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.781583 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.782109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.781614 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 11:13:03.782109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.781787 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 11:13:03.881901 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.881880 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.908802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.908765 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 11:08:04 +0000 UTC" deadline="2028-04-21 16:10:53.834233038 +0000 UTC" Jul 08 11:13:03.908802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.908797 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15676h57m49.925439133s" Jul 08 11:13:03.952081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.952055 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 11:13:03.953474 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.953454 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientMemory" Jul 08 11:13:03.953558 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.953486 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 11:13:03.953558 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:03.953497 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeHasSufficientPID" Jul 08 11:13:03.982790 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:03.982763 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:03.998867 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:03.998838 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e14242d5a8b7e59aec414a566b1b75.slice/crio-88589dddec62997c1b63db0dd99ff1f0806ec629ea5c595d95e139ffa09f2451 WatchSource:0}: Error finding container 88589dddec62997c1b63db0dd99ff1f0806ec629ea5c595d95e139ffa09f2451: Status 404 returned error can't find the container with id 88589dddec62997c1b63db0dd99ff1f0806ec629ea5c595d95e139ffa09f2451 Jul 08 11:13:04.000095 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:04.000057 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2339c7fd5cf13a79b0441cd2cf2b5de7.slice/crio-b80ac69491d77037f2609ed3b4147536ceecbf628d8c5f284d223af102f69699 WatchSource:0}: Error finding container b80ac69491d77037f2609ed3b4147536ceecbf628d8c5f284d223af102f69699: Status 404 returned error can't find the container with id b80ac69491d77037f2609ed3b4147536ceecbf628d8c5f284d223af102f69699 Jul 08 11:13:04.004318 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:04.004298 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 11:13:04.083532 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:04.083503 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:04.184126 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:04.184056 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:04.284632 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:04.284607 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-148.ec2.internal\" not found" Jul 08 11:13:04.306734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:04.306711 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 11:13:04.322679 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:04.322659 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 11:13:05.537555 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.537524 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" Jul 08 11:13:05.583408 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.583381 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 11:13:05.584192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.584179 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" Jul 08 11:13:05.612224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.612199 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 11:13:05.652603 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.652547 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 11:13:05.882868 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:05.882839 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 11:13:06.011207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.010904 2559 apiserver.go:52] "Watching apiserver" Jul 08 11:13:06.018734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.018705 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 11:13:06.020952 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.020922 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-2hf4f","openshift-image-registry/node-ca-blkb4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal","openshift-multus/multus-additional-cni-plugins-x2dww","openshift-multus/multus-b6449","openshift-multus/network-metrics-daemon-46kkp","openshift-network-diagnostics/network-check-target-xlvnq","kube-system/global-pull-secret-syncer-w5f7p","kube-system/konnectivity-agent-6bctf","kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw","openshift-network-operator/iptables-alerter-7rxg4","openshift-ovn-kubernetes/ovnkube-node-fdzdr","openshift-cluster-node-tuning-operator/tuned-964c6"] Jul 08 11:13:06.023430 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.023406 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:06.023540 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.023502 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:06.026704 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.026678 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.029000 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.028982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.031125 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.030913 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.033803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.033207 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b6449" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.034824 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.034839 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035035 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035067 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035257 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035293 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kbhn4\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035535 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.035917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.036764 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.035980 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:06.039075 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.037841 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 11:13:06.039075 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.038521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.041458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.040966 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.043338 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.043321 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.046277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.046262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.046353 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.046333 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:06.047930 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.047905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-netns\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.047943 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-sys\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.048019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.047968 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.048019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.047991 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cnibin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-node-log\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqhk\" (UniqueName: \"kubernetes.io/projected/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-kube-api-access-zzqhk\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048060 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-cnibin\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-log-socket\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysconfig\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-multus\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048224 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-env-overrides\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-host\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxlzr\" (UniqueName: \"kubernetes.io/projected/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-kube-api-access-nxlzr\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-bin\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8132bc83-8fd5-45e4-a58c-947741a8119f-agent-certs\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048379 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-hostroot\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-config\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048431 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-script-lib\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-os-release\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048478 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cni-binary-copy\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-bin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6cx\" (UniqueName: \"kubernetes.io/projected/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-kube-api-access-pn6cx\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.048593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048588 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-tmp\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048654 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048685 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-system-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048726 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-slash\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-k8s-cni-cncf-io\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048881 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-daemon-config\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-etc-kubernetes\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-var-lib-kubelet\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-system-cni-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.048986 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049010 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-modprobe-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-conf\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049058 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.049246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-tuned\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-conf-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049172 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-var-lib-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049223 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-etc-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049248 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovn-node-metrics-cert\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-run\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049298 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-lib-modules\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-ovn\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049351 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-netd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stl46\" (UniqueName: \"kubernetes.io/projected/d9943525-32a2-42a2-8126-23080f05a39e-kube-api-access-stl46\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049401 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8132bc83-8fd5-45e4-a58c-947741a8119f-konnectivity-ca\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049426 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-kubernetes\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-socket-dir-parent\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-multus-certs\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049627 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-kubelet\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049675 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-systemd\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049700 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-kubelet\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-systemd-units\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049761 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-os-release\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049810 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-netns\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h22jd\" (UniqueName: \"kubernetes.io/projected/afb0dca8-0538-4ede-8a26-05a2d37df3a4-kube-api-access-h22jd\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.049874 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-systemd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.050768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.050396 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.062881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.062656 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.062881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.062695 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 11:13:06.063541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.063172 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 11:13:06.063541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.063198 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.063541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.063355 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 11:13:06.063541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.063394 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.063541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.063419 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.064619 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.064818 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.065010 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.065486 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.065690 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.065836 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.066144 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.066341 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.066518 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.066782 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6k5xg\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.066976 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.067150 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.067336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9cpb8\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.067531 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9c4fh\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.068219 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 11:13:06.068672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.068449 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dlhc9\"" Jul 08 11:13:06.069492 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.068707 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-52tzq\"" Jul 08 11:13:06.069492 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.068931 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7vcqt\"" Jul 08 11:13:06.069492 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.069418 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gp5hp\"" Jul 08 11:13:06.069694 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.069568 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w6tdx\"" Jul 08 11:13:06.094217 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.094192 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 11:13:06.094364 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.094343 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:06.102235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.102218 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 11:13:06.139088 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.139056 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 11:13:06.143348 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.143221 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8kcdh" Jul 08 11:13:06.150266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150233 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-netns\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-sys\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cnibin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-node-log\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqhk\" (UniqueName: \"kubernetes.io/projected/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-kube-api-access-zzqhk\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150382 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150365 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-netns\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-cnibin\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-cnibin\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150427 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-log-socket\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150445 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-sys\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150454 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150486 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysconfig\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150534 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-multus\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.150569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-env-overrides\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150576 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c4963efb-d2c2-4328-a5a2-8649d2f85931-iptables-alerter-script\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-host\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150615 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-node-log\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxlzr\" (UniqueName: \"kubernetes.io/projected/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-kube-api-access-nxlzr\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-bin\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150712 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150767 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8132bc83-8fd5-45e4-a58c-947741a8119f-agent-certs\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-hostroot\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150810 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-config\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150833 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-script-lib\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-registration-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.150895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150893 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-os-release\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150903 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-host\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cni-binary-copy\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-log-socket\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150947 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-bin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150970 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.150970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6cx\" (UniqueName: \"kubernetes.io/projected/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-kube-api-access-pn6cx\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151013 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysconfig\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.151307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151038 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-multus\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151344 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-bin\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.151548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151439 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-env-overrides\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.151548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cnibin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151805 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151788 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-cni-bin\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151786 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 11:13:06.151859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151828 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-hostroot\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.151859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151845 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151901 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-socket-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151916 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4963efb-d2c2-4328-a5a2-8649d2f85931-host-slash\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151935 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-tmp\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.151989 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-tmp-dir\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.152024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152037 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-system-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152051 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-slash\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152065 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-hosts-file\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152080 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-k8s-cni-cncf-io\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-daemon-config\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-etc-kubernetes\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-var-lib-kubelet\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152151 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-system-cni-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152175 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152190 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-modprobe-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152186 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-config\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-conf\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152234 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152263 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-tuned\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-conf-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-var-lib-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-etc-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovn-node-metrics-cert\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-run\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152374 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-lib-modules\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-ovn\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152419 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-netd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152433 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-device-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152448 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9rcb\" (UniqueName: \"kubernetes.io/projected/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-kube-api-access-x9rcb\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152470 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/320a10ed-50a0-4a8e-8246-167e3a923848-host\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152484 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/320a10ed-50a0-4a8e-8246-167e3a923848-serviceca\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152495 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovnkube-script-lib\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stl46\" (UniqueName: \"kubernetes.io/projected/d9943525-32a2-42a2-8126-23080f05a39e-kube-api-access-stl46\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152530 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8132bc83-8fd5-45e4-a58c-947741a8119f-konnectivity-ca\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152593 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-os-release\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-kubernetes\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.152921 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-socket-dir-parent\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-multus-certs\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-kubelet\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-ovn-kubernetes\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-systemd\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152784 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-kubelet\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152807 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-systemd-units\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152834 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152858 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8132bc83-8fd5-45e4-a58c-947741a8119f-konnectivity-ca\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152862 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbmgw\" (UniqueName: \"kubernetes.io/projected/c1294dae-1034-434b-8c58-f57746855151-kube-api-access-bbmgw\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-kubelet-config\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-os-release\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152953 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-netns\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152981 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-sys-fs\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.152990 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-run-netns\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h22jd\" (UniqueName: \"kubernetes.io/projected/afb0dca8-0538-4ede-8a26-05a2d37df3a4-kube-api-access-h22jd\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.153862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153169 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-etc-kubernetes\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153210 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-var-lib-kubelet\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153240 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-system-cni-dir\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153541 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153562 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-daemon-config\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153590 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-cni-binary-copy\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-systemd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153626 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153747 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-system-cni-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153783 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-slash\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-k8s-cni-cncf-io\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153902 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-conf-dir\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.153918 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-lib-modules\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.153963 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154011 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-sysctl-conf\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.154018 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:06.65399407 +0000 UTC m=+3.035640818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154053 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-cni-netd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.155444 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154177 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-var-lib-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154223 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-etc-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9943525-32a2-42a2-8126-23080f05a39e-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154529 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9943525-32a2-42a2-8126-23080f05a39e-os-release\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-host-kubelet\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154603 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-systemd\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-var-lib-kubelet\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-systemd-units\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154738 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-kubernetes\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154744 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-openvswitch\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2xgm\" (UniqueName: \"kubernetes.io/projected/c4963efb-d2c2-4328-a5a2-8649d2f85931-kube-api-access-r2xgm\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv28v\" (UniqueName: \"kubernetes.io/projected/320a10ed-50a0-4a8e-8246-167e3a923848-kube-api-access-zv28v\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154802 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-run\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-dbus\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-systemd\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154881 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-run-ovn\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154912 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-multus-socket-dir-parent\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.156247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154951 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-modprobe-d\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.157574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.154957 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-host-run-multus-certs\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.157574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.156004 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-tmp\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.157574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.156284 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8132bc83-8fd5-45e4-a58c-947741a8119f-agent-certs\") pod \"konnectivity-agent-6bctf\" (UID: \"8132bc83-8fd5-45e4-a58c-947741a8119f\") " pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.157574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.156649 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-etc-tuned\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.157844 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.157740 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-ovn-node-metrics-cert\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.158699 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.158120 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8kcdh" Jul 08 11:13:06.165262 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.165243 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:06.165362 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.165268 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:06.165362 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.165282 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:06.165362 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.165341 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:06.665324214 +0000 UTC m=+3.046970984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:06.168521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.168494 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stl46\" (UniqueName: \"kubernetes.io/projected/d9943525-32a2-42a2-8126-23080f05a39e-kube-api-access-stl46\") pod \"multus-additional-cni-plugins-x2dww\" (UID: \"d9943525-32a2-42a2-8126-23080f05a39e\") " pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.169553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.169527 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h22jd\" (UniqueName: \"kubernetes.io/projected/afb0dca8-0538-4ede-8a26-05a2d37df3a4-kube-api-access-h22jd\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.170022 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.169923 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" event={"ID":"2339c7fd5cf13a79b0441cd2cf2b5de7","Type":"ContainerStarted","Data":"b80ac69491d77037f2609ed3b4147536ceecbf628d8c5f284d223af102f69699"} Jul 08 11:13:06.170570 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.170174 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6cx\" (UniqueName: \"kubernetes.io/projected/7a48599e-3b7f-4a6c-bf9f-a451c1d3836b-kube-api-access-pn6cx\") pod \"multus-b6449\" (UID: \"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b\") " pod="openshift-multus/multus-b6449" Jul 08 11:13:06.170570 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.170528 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqhk\" (UniqueName: \"kubernetes.io/projected/2e4e0499-2fb1-4d75-b0fd-8989299bc68b-kube-api-access-zzqhk\") pod \"ovnkube-node-fdzdr\" (UID: \"2e4e0499-2fb1-4d75-b0fd-8989299bc68b\") " pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.170851 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.170834 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxlzr\" (UniqueName: \"kubernetes.io/projected/0015b57d-e97c-4a56-9276-6cd7bafdc8b0-kube-api-access-nxlzr\") pod \"tuned-964c6\" (UID: \"0015b57d-e97c-4a56-9276-6cd7bafdc8b0\") " pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.172169 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.172115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" event={"ID":"97e14242d5a8b7e59aec414a566b1b75","Type":"ContainerStarted","Data":"88589dddec62997c1b63db0dd99ff1f0806ec629ea5c595d95e139ffa09f2451"} Jul 08 11:13:06.255141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/320a10ed-50a0-4a8e-8246-167e3a923848-host\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/320a10ed-50a0-4a8e-8246-167e3a923848-serviceca\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255238 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255243 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/320a10ed-50a0-4a8e-8246-167e3a923848-host\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbmgw\" (UniqueName: \"kubernetes.io/projected/c1294dae-1034-434b-8c58-f57746855151-kube-api-access-bbmgw\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255288 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-kubelet-config\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-sys-fs\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2xgm\" (UniqueName: \"kubernetes.io/projected/c4963efb-d2c2-4328-a5a2-8649d2f85931-kube-api-access-r2xgm\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv28v\" (UniqueName: \"kubernetes.io/projected/320a10ed-50a0-4a8e-8246-167e3a923848-kube-api-access-zv28v\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255391 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-kubelet-config\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-dbus\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c4963efb-d2c2-4328-a5a2-8649d2f85931-iptables-alerter-script\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255504 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-registration-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-socket-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255579 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4963efb-d2c2-4328-a5a2-8649d2f85931-host-slash\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.255626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-tmp-dir\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255659 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/320a10ed-50a0-4a8e-8246-167e3a923848-serviceca\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255666 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-hosts-file\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255729 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-sys-fs\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-device-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255776 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255775 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c4963efb-d2c2-4328-a5a2-8649d2f85931-host-slash\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255791 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9rcb\" (UniqueName: \"kubernetes.io/projected/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-kube-api-access-x9rcb\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-registration-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255796 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-socket-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255864 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b358f565-efb3-4820-b6df-d5aed770cbb2-dbus\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.255865 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-hosts-file\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.256031 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c1294dae-1034-434b-8c58-f57746855151-device-dir\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.256053 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-tmp-dir\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.256091 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.256082 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:06.256686 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.256135 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:06.756118552 +0000 UTC m=+3.137765302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:06.256686 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.256261 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c4963efb-d2c2-4328-a5a2-8649d2f85931-iptables-alerter-script\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.292494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.292425 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9rcb\" (UniqueName: \"kubernetes.io/projected/ac3d9ce1-d290-4f67-80d2-98e05b218cc2-kube-api-access-x9rcb\") pod \"node-resolver-2hf4f\" (UID: \"ac3d9ce1-d290-4f67-80d2-98e05b218cc2\") " pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.295047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.295020 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv28v\" (UniqueName: \"kubernetes.io/projected/320a10ed-50a0-4a8e-8246-167e3a923848-kube-api-access-zv28v\") pod \"node-ca-blkb4\" (UID: \"320a10ed-50a0-4a8e-8246-167e3a923848\") " pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.295161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.295104 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2xgm\" (UniqueName: \"kubernetes.io/projected/c4963efb-d2c2-4328-a5a2-8649d2f85931-kube-api-access-r2xgm\") pod \"iptables-alerter-7rxg4\" (UID: \"c4963efb-d2c2-4328-a5a2-8649d2f85931\") " pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.297453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.297416 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbmgw\" (UniqueName: \"kubernetes.io/projected/c1294dae-1034-434b-8c58-f57746855151-kube-api-access-bbmgw\") pod \"aws-ebs-csi-driver-node-mp7bw\" (UID: \"c1294dae-1034-434b-8c58-f57746855151\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.340209 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.340178 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:06.349620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.349604 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-964c6" Jul 08 11:13:06.349889 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.349870 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4e0499_2fb1_4d75_b0fd_8989299bc68b.slice/crio-03a7361c6edb7636bd5e89d5e9d4eb7f7837439abfdc21949133614f74c215d9 WatchSource:0}: Error finding container 03a7361c6edb7636bd5e89d5e9d4eb7f7837439abfdc21949133614f74c215d9: Status 404 returned error can't find the container with id 03a7361c6edb7636bd5e89d5e9d4eb7f7837439abfdc21949133614f74c215d9 Jul 08 11:13:06.357678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.357624 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2dww" Jul 08 11:13:06.360130 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.360107 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0015b57d_e97c_4a56_9276_6cd7bafdc8b0.slice/crio-791ebe5a65e5f723478b0231832a6838300db43a4a58bbe4a8135b89f11833cf WatchSource:0}: Error finding container 791ebe5a65e5f723478b0231832a6838300db43a4a58bbe4a8135b89f11833cf: Status 404 returned error can't find the container with id 791ebe5a65e5f723478b0231832a6838300db43a4a58bbe4a8135b89f11833cf Jul 08 11:13:06.364754 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.364734 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b6449" Jul 08 11:13:06.367627 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.367231 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9943525_32a2_42a2_8126_23080f05a39e.slice/crio-74bf3fed3e6c0a9330f433198beb51041444610da7c5c4cce7cacef1ac05f04e WatchSource:0}: Error finding container 74bf3fed3e6c0a9330f433198beb51041444610da7c5c4cce7cacef1ac05f04e: Status 404 returned error can't find the container with id 74bf3fed3e6c0a9330f433198beb51041444610da7c5c4cce7cacef1ac05f04e Jul 08 11:13:06.372942 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.372920 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:06.375261 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.375236 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a48599e_3b7f_4a6c_bf9f_a451c1d3836b.slice/crio-2982b8a9d3c50481f67a6ddff1d88e2d8e03201c3174a3c6234206c5ec7fbcf2 WatchSource:0}: Error finding container 2982b8a9d3c50481f67a6ddff1d88e2d8e03201c3174a3c6234206c5ec7fbcf2: Status 404 returned error can't find the container with id 2982b8a9d3c50481f67a6ddff1d88e2d8e03201c3174a3c6234206c5ec7fbcf2 Jul 08 11:13:06.381035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.380981 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2hf4f" Jul 08 11:13:06.383420 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.383394 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8132bc83_8fd5_45e4_a58c_947741a8119f.slice/crio-20f322d92b2ebad7980fc1755395363166dffba4d72d3841c88c91b595a030f0 WatchSource:0}: Error finding container 20f322d92b2ebad7980fc1755395363166dffba4d72d3841c88c91b595a030f0: Status 404 returned error can't find the container with id 20f322d92b2ebad7980fc1755395363166dffba4d72d3841c88c91b595a030f0 Jul 08 11:13:06.387309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.387286 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-blkb4" Jul 08 11:13:06.390205 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.390180 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3d9ce1_d290_4f67_80d2_98e05b218cc2.slice/crio-2a3acae901695930cde0d2e38cd027008ec596b8491901c22ddc5840f2c9e35f WatchSource:0}: Error finding container 2a3acae901695930cde0d2e38cd027008ec596b8491901c22ddc5840f2c9e35f: Status 404 returned error can't find the container with id 2a3acae901695930cde0d2e38cd027008ec596b8491901c22ddc5840f2c9e35f Jul 08 11:13:06.394237 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.394221 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" Jul 08 11:13:06.395782 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.395759 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320a10ed_50a0_4a8e_8246_167e3a923848.slice/crio-8c704cbbb49fc75e1d3cddf34ec9e8ecbfe8632ac3a6b63df6f13629cb293bc2 WatchSource:0}: Error finding container 8c704cbbb49fc75e1d3cddf34ec9e8ecbfe8632ac3a6b63df6f13629cb293bc2: Status 404 returned error can't find the container with id 8c704cbbb49fc75e1d3cddf34ec9e8ecbfe8632ac3a6b63df6f13629cb293bc2 Jul 08 11:13:06.400907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.400886 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7rxg4" Jul 08 11:13:06.408285 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.408260 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1294dae_1034_434b_8c58_f57746855151.slice/crio-586f23c03fa0b3985a1e422a5423e2ca267e54f23cb6e685b16f587753a18f33 WatchSource:0}: Error finding container 586f23c03fa0b3985a1e422a5423e2ca267e54f23cb6e685b16f587753a18f33: Status 404 returned error can't find the container with id 586f23c03fa0b3985a1e422a5423e2ca267e54f23cb6e685b16f587753a18f33 Jul 08 11:13:06.411392 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:06.411369 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4963efb_d2c2_4328_a5a2_8649d2f85931.slice/crio-5cdac76d5312b93916a5896b6e19493fc02277d2f583acf8b94466a98c3ec5e4 WatchSource:0}: Error finding container 5cdac76d5312b93916a5896b6e19493fc02277d2f583acf8b94466a98c3ec5e4: Status 404 returned error can't find the container with id 5cdac76d5312b93916a5896b6e19493fc02277d2f583acf8b94466a98c3ec5e4 Jul 08 11:13:06.658592 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.658494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:06.658772 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.658684 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:06.658772 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.658764 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:07.658742989 +0000 UTC m=+4.040389754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:06.759273 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.759216 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:06.759439 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:06.759301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:06.759439 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759401 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:06.759565 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759473 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:07.759455503 +0000 UTC m=+4.141102255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:06.759565 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759411 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:06.759565 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759503 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:06.759565 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759517 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:06.759778 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:06.759573 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:07.759558552 +0000 UTC m=+4.141205300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:07.162390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.162344 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 11:08:06 +0000 UTC" deadline="2028-01-31 16:45:27.616235917 +0000 UTC" Jul 08 11:13:07.162390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.162384 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13733h32m20.453855529s" Jul 08 11:13:07.175164 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.175123 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"03a7361c6edb7636bd5e89d5e9d4eb7f7837439abfdc21949133614f74c215d9"} Jul 08 11:13:07.176352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.176327 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7rxg4" event={"ID":"c4963efb-d2c2-4328-a5a2-8649d2f85931","Type":"ContainerStarted","Data":"5cdac76d5312b93916a5896b6e19493fc02277d2f583acf8b94466a98c3ec5e4"} Jul 08 11:13:07.177650 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.177607 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-blkb4" event={"ID":"320a10ed-50a0-4a8e-8246-167e3a923848","Type":"ContainerStarted","Data":"8c704cbbb49fc75e1d3cddf34ec9e8ecbfe8632ac3a6b63df6f13629cb293bc2"} Jul 08 11:13:07.178940 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.178916 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b6449" event={"ID":"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b","Type":"ContainerStarted","Data":"2982b8a9d3c50481f67a6ddff1d88e2d8e03201c3174a3c6234206c5ec7fbcf2"} Jul 08 11:13:07.180137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.180110 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-964c6" event={"ID":"0015b57d-e97c-4a56-9276-6cd7bafdc8b0","Type":"ContainerStarted","Data":"791ebe5a65e5f723478b0231832a6838300db43a4a58bbe4a8135b89f11833cf"} Jul 08 11:13:07.181343 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.181320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" event={"ID":"c1294dae-1034-434b-8c58-f57746855151","Type":"ContainerStarted","Data":"586f23c03fa0b3985a1e422a5423e2ca267e54f23cb6e685b16f587753a18f33"} Jul 08 11:13:07.182476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.182456 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2hf4f" event={"ID":"ac3d9ce1-d290-4f67-80d2-98e05b218cc2","Type":"ContainerStarted","Data":"2a3acae901695930cde0d2e38cd027008ec596b8491901c22ddc5840f2c9e35f"} Jul 08 11:13:07.183579 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.183557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bctf" event={"ID":"8132bc83-8fd5-45e4-a58c-947741a8119f","Type":"ContainerStarted","Data":"20f322d92b2ebad7980fc1755395363166dffba4d72d3841c88c91b595a030f0"} Jul 08 11:13:07.184738 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.184707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerStarted","Data":"74bf3fed3e6c0a9330f433198beb51041444610da7c5c4cce7cacef1ac05f04e"} Jul 08 11:13:07.676113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.675511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:07.676113 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.675705 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:07.676113 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.675767 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:09.675749559 +0000 UTC m=+6.057396314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.777673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:07.777736 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.777927 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.777947 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.777960 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.778020 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:09.778001268 +0000 UTC m=+6.159648031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.778399 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:07.778486 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:07.778447 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:09.778432733 +0000 UTC m=+6.160079496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:08.163304 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 11:08:06 +0000 UTC" deadline="2028-03-02 05:56:35.826639553 +0000 UTC" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:08.163526 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14466h43m27.663118564s" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:08.163532 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:08.163486 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:08.163699 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:08.163857 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:08.163923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:08.166091 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:08.164021 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:09.207607 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.207561 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" event={"ID":"2339c7fd5cf13a79b0441cd2cf2b5de7","Type":"ContainerStarted","Data":"d056017781019f2a7173baab1248702bd063a8430e447841545fb5132f2b1a25"} Jul 08 11:13:09.231085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.231048 2559 generic.go:356] "Generic (PLEG): container finished" podID="97e14242d5a8b7e59aec414a566b1b75" containerID="f74b105cb64d32ade8a9ff6a3cb31ef9d6219ab024cdc2c89fa5200cb1eddb2c" exitCode=0 Jul 08 11:13:09.231240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.231106 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" event={"ID":"97e14242d5a8b7e59aec414a566b1b75","Type":"ContainerDied","Data":"f74b105cb64d32ade8a9ff6a3cb31ef9d6219ab024cdc2c89fa5200cb1eddb2c"} Jul 08 11:13:09.317081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.317018 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-148.ec2.internal" podStartSLOduration=4.316996055 podStartE2EDuration="4.316996055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:09.266035992 +0000 UTC m=+5.647682765" watchObservedRunningTime="2026-07-08 11:13:09.316996055 +0000 UTC m=+5.698642827" Jul 08 11:13:09.696627 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.696595 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:09.696946 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.696805 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:09.696946 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.696879 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:13.696857565 +0000 UTC m=+10.078504336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:09.797760 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.797681 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:09.797760 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:09.797746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:09.797973 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.797900 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:09.797973 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.797918 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:09.797973 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.797930 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:09.798131 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.797987 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:13.797968044 +0000 UTC m=+10.179614808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:09.798407 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.798385 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:09.798516 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:09.798441 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:13.798426208 +0000 UTC m=+10.180072973 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:10.175710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:10.174992 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:10.175710 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:10.175116 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:10.175710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:10.175511 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:10.175710 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:10.175602 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:10.176318 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:10.176148 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:10.176318 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:10.176271 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:10.236767 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:10.236696 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" event={"ID":"97e14242d5a8b7e59aec414a566b1b75","Type":"ContainerStarted","Data":"1b0802902663127b47fd0e0bbd16280fedc51e45d1e402ab6358022744ec0b8f"} Jul 08 11:13:12.163554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:12.162907 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:12.163554 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:12.163038 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:12.163554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:12.163422 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:12.173304 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:12.164555 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:12.173304 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:12.172805 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:12.173304 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:12.172958 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:13.729080 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:13.729040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:13.729494 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.729215 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:13.729494 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.729304 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:21.729279672 +0000 UTC m=+18.110926430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:13.829911 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:13.829869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:13.830078 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:13.829971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:13.830152 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830091 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:13.830152 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830148 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:21.830131069 +0000 UTC m=+18.211777822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:13.830553 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830536 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:13.830553 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830554 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:13.830700 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830563 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:13.830700 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:13.830598 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:21.830587319 +0000 UTC m=+18.212234068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:14.164827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:14.164749 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:14.164827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:14.164749 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:14.165846 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:14.165808 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:14.166069 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:14.166039 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:14.166174 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:14.166148 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:14.166296 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:14.166249 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:16.163447 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:16.163412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:16.163447 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:16.163430 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:16.163968 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:16.163412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:16.163968 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:16.163577 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:16.163968 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:16.163653 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:16.164509 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:16.164089 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:18.163504 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.163243 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:18.163872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.163243 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:18.163872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.163243 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:18.163872 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:18.163682 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:18.163872 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:18.163736 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:18.163872 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:18.163619 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:18.256462 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.256416 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-blkb4" event={"ID":"320a10ed-50a0-4a8e-8246-167e3a923848","Type":"ContainerStarted","Data":"b63f165311a56d99a6e84ec6f200534afdd758d449958c9ecf57f26e2fdaf559"} Jul 08 11:13:18.258121 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.258090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-964c6" event={"ID":"0015b57d-e97c-4a56-9276-6cd7bafdc8b0","Type":"ContainerStarted","Data":"b9062e9762bd7c5f38fa8e173f9205923c9ca26076fd77fec34224bddc692307"} Jul 08 11:13:18.259858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.259830 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" event={"ID":"c1294dae-1034-434b-8c58-f57746855151","Type":"ContainerStarted","Data":"b72fc5800798c05dc4321706d67024b479fb703cf7ed08469cf65f39c2ba111f"} Jul 08 11:13:18.261889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.261545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2hf4f" event={"ID":"ac3d9ce1-d290-4f67-80d2-98e05b218cc2","Type":"ContainerStarted","Data":"c5a31508f1fad0c2bdb985233caa051ccc2a6a99d0abc2ef27f71dcedaf26d5e"} Jul 08 11:13:18.263628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.263286 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bctf" event={"ID":"8132bc83-8fd5-45e4-a58c-947741a8119f","Type":"ContainerStarted","Data":"7021014b26ff058bc1e3282593bd774917fed03ad0fa365d852d687df8136dd0"} Jul 08 11:13:18.265676 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.265656 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="bf10dc3c942a38e66a5130907ea0523d1b693fe02a42733fdc7b3780e1bdeebf" exitCode=0 Jul 08 11:13:18.265774 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.265689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"bf10dc3c942a38e66a5130907ea0523d1b693fe02a42733fdc7b3780e1bdeebf"} Jul 08 11:13:18.282172 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.282126 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-148.ec2.internal" podStartSLOduration=13.282096355 podStartE2EDuration="13.282096355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:10.264514559 +0000 UTC m=+6.646161363" watchObservedRunningTime="2026-07-08 11:13:18.282096355 +0000 UTC m=+14.663743119" Jul 08 11:13:18.283113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.283077 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-blkb4" podStartSLOduration=3.335494286 podStartE2EDuration="14.283064913s" totalImagesPullingTime="10.947570627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:18.281501313 +0000 UTC m=+14.663148085" watchObservedRunningTime="2026-07-08 11:13:18.283064913 +0000 UTC m=+14.664711686" Jul 08 11:13:18.369604 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.369310 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-964c6" podStartSLOduration=3.117985369 podStartE2EDuration="14.369289647s" totalImagesPullingTime="11.251304278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:18.368530939 +0000 UTC m=+14.750177712" watchObservedRunningTime="2026-07-08 11:13:18.369289647 +0000 UTC m=+14.750936438" Jul 08 11:13:18.369604 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.369460 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2hf4f" podStartSLOduration=3.416787534 podStartE2EDuration="14.369450369s" totalImagesPullingTime="10.952662835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:18.331529122 +0000 UTC m=+14.713175896" watchObservedRunningTime="2026-07-08 11:13:18.369450369 +0000 UTC m=+14.751097147" Jul 08 11:13:18.443237 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:18.443131 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6bctf" podStartSLOduration=3.458629862 podStartE2EDuration="14.443091378s" totalImagesPullingTime="10.984461516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:18.442951404 +0000 UTC m=+14.824598176" watchObservedRunningTime="2026-07-08 11:13:18.443091378 +0000 UTC m=+14.824738151" Jul 08 11:13:19.270792 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:19.270750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7rxg4" event={"ID":"c4963efb-d2c2-4328-a5a2-8649d2f85931","Type":"ContainerStarted","Data":"5dcbbd18246b6266afee240f96f815a03b633dce0e2198d42c5b102bc57c598c"} Jul 08 11:13:20.163471 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:20.163437 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:20.163671 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:20.163451 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:20.163671 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:20.163571 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:20.163671 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:20.163576 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:20.163833 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:20.163669 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:20.163833 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:20.163742 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:20.434965 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:20.434868 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:21.351998 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.351956 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:21.352850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.352820 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:21.378536 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.378470 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7rxg4" podStartSLOduration=6.426121518 podStartE2EDuration="17.378448881s" totalImagesPullingTime="10.952327363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:19.292938175 +0000 UTC m=+15.674584947" watchObservedRunningTime="2026-07-08 11:13:21.378448881 +0000 UTC m=+17.760095653" Jul 08 11:13:21.794518 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.794481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:21.795207 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.794612 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:21.795207 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.794694 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:37.794674354 +0000 UTC m=+34.176321119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:21.895227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.895189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:21.895418 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:21.895287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:21.895418 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895384 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:21.895418 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895390 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:21.895418 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895406 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:21.895418 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895416 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:21.895651 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895456 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:37.895437652 +0000 UTC m=+34.277084403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:21.895651 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:21.895475 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:37.895465432 +0000 UTC m=+34.277112184 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:22.163748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:22.163658 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:22.163748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:22.163691 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:22.163936 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:22.163790 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:22.163936 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:22.163854 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:22.163936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:22.163920 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:22.164111 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:22.164011 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:22.277967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:22.277936 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6bctf" Jul 08 11:13:24.164524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:24.164491 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:24.165154 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:24.164581 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:24.165154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:24.164975 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:24.165154 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:24.165083 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:24.165154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:24.165131 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:24.165350 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:24.165193 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:26.163416 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:26.163368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:26.163416 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:26.163395 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:26.164074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:26.163385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:26.164074 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:26.163504 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:26.164074 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:26.163628 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:26.164074 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:26.163740 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:27.560950 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:27.560801 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 11:13:28.118871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.118581 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T11:13:27.560944563Z","UUID":"a875637d-1013-4343-b99f-1de489b1929c","Handler":null,"Name":"","Endpoint":""} Jul 08 11:13:28.120090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.120059 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 11:13:28.120090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.120095 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.163087 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:28.163214 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.163271 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:28.163368 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.163278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:28.163721 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:28.163558 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:28.292149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.292092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b6449" event={"ID":"7a48599e-3b7f-4a6c-bf9f-a451c1d3836b","Type":"ContainerStarted","Data":"a917f82ad417f1e8bd23dbeb72b0e243e92a8ac62e223370e351e390db5f312c"} Jul 08 11:13:28.294889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.294859 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" event={"ID":"c1294dae-1034-434b-8c58-f57746855151","Type":"ContainerStarted","Data":"8cec1668bbacbd19d6404065ebee12a9ca55a7345e71984bbd15d3af6f008a79"} Jul 08 11:13:28.299325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"5eb7adef90a654b0e89971e1e780cd2f4e791a254b21a2b6023262c054e5e031"} Jul 08 11:13:28.299498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"aad26567a90e23fe79811c5db2ab75e58cc23324c52f2355cb398c1a3e2c3b5f"} Jul 08 11:13:28.299498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299415 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"9cae5bb22171a29864eef491dacffc25444c02f6ede50c02bc73e829bf2c5175"} Jul 08 11:13:28.299498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299462 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"c273e4f7d1e7b2b9782cd9895bbdef47c707114d09ea4c318c6a2bbf5fd11c20"} Jul 08 11:13:28.299664 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"613ddea2ba111ed29758d0ddb84a94545e2a602ce306c35ea7902ae9374b2177"} Jul 08 11:13:28.299664 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.299542 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"86c62cd9511de34d56b03fc06dd8c66d00558cd336f1318525b0badb5e260b49"} Jul 08 11:13:28.316461 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:28.316413 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b6449" podStartSLOduration=3.293499023 podStartE2EDuration="24.316399221s" totalImagesPullingTime="21.022900198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:28.315231251 +0000 UTC m=+24.696878022" watchObservedRunningTime="2026-07-08 11:13:28.316399221 +0000 UTC m=+24.698046003" Jul 08 11:13:30.163279 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:30.163254 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:30.163279 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:30.163278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:30.163850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:30.163254 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:30.163850 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:30.163369 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:30.163850 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:30.163431 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:30.163850 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:30.163486 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:31.307761 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:31.307713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"59ecdf4cd1b93a0726f8005314ab7258da3a837ef8708955c2c59995efb28fa6"} Jul 08 11:13:31.309830 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:31.309804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" event={"ID":"c1294dae-1034-434b-8c58-f57746855151","Type":"ContainerStarted","Data":"53af4ee48d452051e0a242b2e0c3699ad3d92e268c891f17b45edf05428a35e7"} Jul 08 11:13:31.311494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:31.311473 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="70639857f26e35703a33cdea98643b05514d27ef073d0092b64f741f7a3f35b4" exitCode=0 Jul 08 11:13:31.311577 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:31.311502 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"70639857f26e35703a33cdea98643b05514d27ef073d0092b64f741f7a3f35b4"} Jul 08 11:13:31.336960 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:31.336917 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp7bw" podStartSLOduration=3.9895732600000002 podStartE2EDuration="27.33690371s" totalImagesPullingTime="23.34733045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:31.33647444 +0000 UTC m=+27.718121211" watchObservedRunningTime="2026-07-08 11:13:31.33690371 +0000 UTC m=+27.718550480" Jul 08 11:13:32.163408 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:32.163377 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:32.163675 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:32.163489 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:32.163675 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:32.163494 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:32.163675 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:32.163624 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:32.163794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:32.163689 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:32.163794 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:32.163783 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:33.317553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.317364 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="288aa471a63423f1171f311419d1d970f4305e0a1fb621522810a36db9a316b1" exitCode=0 Jul 08 11:13:33.318277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.317438 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"288aa471a63423f1171f311419d1d970f4305e0a1fb621522810a36db9a316b1"} Jul 08 11:13:33.321479 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.321450 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" event={"ID":"2e4e0499-2fb1-4d75-b0fd-8989299bc68b","Type":"ContainerStarted","Data":"d955e4b996fa10ae8cedf8cd8a506507c9ed7e9ee0c42627eec2d17a1186f777"} Jul 08 11:13:33.321684 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.321669 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:33.321750 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.321738 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:33.339499 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.339467 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:33.381927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:33.381865 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" podStartSLOduration=8.274317272 podStartE2EDuration="29.38184462s" totalImagesPullingTime="21.107527348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:33.381588598 +0000 UTC m=+29.763235369" watchObservedRunningTime="2026-07-08 11:13:33.38184462 +0000 UTC m=+29.763491392" Jul 08 11:13:34.164072 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.164037 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:34.164245 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.164113 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:34.164245 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.164131 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:34.164245 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.164227 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:34.164366 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.164291 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:34.164401 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.164383 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:34.323948 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.323923 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:34.339283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.339258 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:13:34.971818 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.971783 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlvnq"] Jul 08 11:13:34.971974 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.971942 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:34.972059 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.972038 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:34.974711 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.974691 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w5f7p"] Jul 08 11:13:34.974820 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.974788 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:34.974927 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.974901 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:34.975983 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.975960 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46kkp"] Jul 08 11:13:34.976081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:34.976067 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:34.976206 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:34.976182 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:35.327920 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:35.327822 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="676fe8d75007a3b38bcb1aecd4110e010749fc4031a47bb9326fd8f285119d52" exitCode=0 Jul 08 11:13:35.327920 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:35.327899 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"676fe8d75007a3b38bcb1aecd4110e010749fc4031a47bb9326fd8f285119d52"} Jul 08 11:13:36.163767 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:36.163730 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:36.163935 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:36.163866 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:37.162801 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:37.162764 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:37.163145 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.162878 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:37.163145 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:37.162933 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:37.163145 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.163028 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:37.811786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:37.811752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:37.811980 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.811909 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:37.811980 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.811974 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs podName:afb0dca8-0538-4ede-8a26-05a2d37df3a4 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:09.811958974 +0000 UTC m=+66.193605722 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs") pod "network-metrics-daemon-46kkp" (UID: "afb0dca8-0538-4ede-8a26-05a2d37df3a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 11:13:37.912231 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:37.912200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:37.912430 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:37.912262 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:37.912430 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912376 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:37.912430 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912397 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 11:13:37.912430 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912420 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 11:13:37.912430 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912433 2559 projected.go:196] Error preparing data for projected volume kube-api-access-4fzrh for pod openshift-network-diagnostics/network-check-target-xlvnq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:37.912625 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912443 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret podName:b358f565-efb3-4820-b6df-d5aed770cbb2 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:09.912425571 +0000 UTC m=+66.294072325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret") pod "global-pull-secret-syncer-w5f7p" (UID: "b358f565-efb3-4820-b6df-d5aed770cbb2") : object "kube-system"/"original-pull-secret" not registered Jul 08 11:13:37.912625 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:37.912482 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh podName:4d73a979-8fb1-47e6-93ef-bacda432632c nodeName:}" failed. No retries permitted until 2026-07-08 11:14:09.912466392 +0000 UTC m=+66.294113158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4fzrh" (UniqueName: "kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh") pod "network-check-target-xlvnq" (UID: "4d73a979-8fb1-47e6-93ef-bacda432632c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 11:13:38.163763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:38.163673 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:38.164189 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:38.163804 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:39.163886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:39.163847 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:39.164338 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:39.163852 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:39.164338 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:39.164007 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46kkp" podUID="afb0dca8-0538-4ede-8a26-05a2d37df3a4" Jul 08 11:13:39.164338 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:39.164068 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlvnq" podUID="4d73a979-8fb1-47e6-93ef-bacda432632c" Jul 08 11:13:40.163903 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:40.163869 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:40.164308 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:40.164008 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w5f7p" podUID="b358f565-efb3-4820-b6df-d5aed770cbb2" Jul 08 11:13:40.946835 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:40.946803 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-148.ec2.internal" event="NodeReady" Jul 08 11:13:40.946997 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:40.946976 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 11:13:41.034795 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.034755 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2zrxz"] Jul 08 11:13:41.039966 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.039940 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.041341 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.041099 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl"] Jul 08 11:13:41.045070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.045051 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:13:41.045188 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.045174 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.048221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.048199 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds"] Jul 08 11:13:41.048330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.048319 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.048707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.048566 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 11:13:41.049628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.049199 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-g2k69\"" Jul 08 11:13:41.051801 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.051778 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf"] Jul 08 11:13:41.052550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.052532 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" Jul 08 11:13:41.052872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.052851 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.052989 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.052972 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.056487 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.056466 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zpsrs"] Jul 08 11:13:41.056654 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.056618 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.059883 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.059864 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.062079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.062060 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 11:13:41.070407 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.070384 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2zrxz"] Jul 08 11:13:41.071287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.071257 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 11:13:41.071408 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.071318 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 11:13:41.071598 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.071577 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c5jq9\"" Jul 08 11:13:41.071723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.071655 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.071723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.071654 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2lwxc\"" Jul 08 11:13:41.072143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072038 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 11:13:41.072143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072049 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 11:13:41.072143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072112 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tch2c\"" Jul 08 11:13:41.072143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072122 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 11:13:41.072143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072142 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072234 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072251 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072271 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072316 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072328 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072356 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072382 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 11:13:41.072421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072394 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-d9598\"" Jul 08 11:13:41.072793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072466 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.072793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072488 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.072793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072535 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 11:13:41.072793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072583 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.072793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.072586 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4bbvl\"" Jul 08 11:13:41.074987 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.074967 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 11:13:41.087517 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.087450 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl"] Jul 08 11:13:41.087831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.087813 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds"] Jul 08 11:13:41.113900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.113841 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zpsrs"] Jul 08 11:13:41.114785 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.114758 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lkn52"] Jul 08 11:13:41.118365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.118345 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d96df899f-2wt7h"] Jul 08 11:13:41.118553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.118518 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.121432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.121410 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb"] Jul 08 11:13:41.121598 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.121580 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cp6jk\"" Jul 08 11:13:41.121822 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.121586 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.126884 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.125956 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 11:13:41.127681 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.127658 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d"] Jul 08 11:13:41.128222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.128200 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.131364 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.131345 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh"] Jul 08 11:13:41.131477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.131461 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.134420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134401 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 11:13:41.134579 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134555 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9hns5"] Jul 08 11:13:41.134689 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134468 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.134689 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134678 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.134823 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134446 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 11:13:41.134931 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.134540 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 11:13:41.135051 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.135034 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.135618 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.135522 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p8r5m\"" Jul 08 11:13:41.135750 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.135685 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.135750 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.135712 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 11:13:41.136230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136208 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52p4m\" (UniqueName: \"kubernetes.io/projected/37ba2488-948f-43c4-b05e-332f112544b7-kube-api-access-52p4m\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.136326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.136326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.136326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136302 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136478 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136280 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 11:13:41.136478 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136355 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-tmp\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.136478 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136381 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136478 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136402 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.136478 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136481 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0393330b-f585-4071-afa5-7740af485277-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-snapshots\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136713 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r7795\"" Jul 08 11:13:41.136731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136727 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mwcg\" (UniqueName: \"kubernetes.io/projected/056b73f6-ff1d-49f9-b96b-d17a2c513b0e-kube-api-access-4mwcg\") pod \"volume-data-source-validator-5d456fdbcd-8vjds\" (UID: \"056b73f6-ff1d-49f9-b96b-d17a2c513b0e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" Jul 08 11:13:41.137028 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136762 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0393330b-f585-4071-afa5-7740af485277-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.137028 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ldsh\" (UniqueName: \"kubernetes.io/projected/0393330b-f585-4071-afa5-7740af485277-kube-api-access-6ldsh\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.137028 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136828 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.137028 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136851 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx8m2\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.137028 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.136880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37ba2488-948f-43c4-b05e-332f112544b7-serving-cert\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.137571 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.137555 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:13:41.137669 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.137594 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf"] Jul 08 11:13:41.137732 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.137691 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.140731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.140713 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" Jul 08 11:13:41.148501 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.148479 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.148652 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.148617 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 11:13:41.149013 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.148994 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rmg2f\"" Jul 08 11:13:41.149799 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.149779 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.149909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.149893 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 11:13:41.149974 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.149930 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 11:13:41.150070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.150054 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 11:13:41.150138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.150088 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 11:13:41.150790 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.150770 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf"] Jul 08 11:13:41.151581 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151441 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 11:13:41.151581 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151455 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 11:13:41.151581 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151473 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.151581 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151520 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.151794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151622 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 11:13:41.151794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151691 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 11:13:41.151794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.151703 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wkr92\"" Jul 08 11:13:41.156458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.156436 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bg7hn\"" Jul 08 11:13:41.156556 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.156509 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mwwd4\"" Jul 08 11:13:41.163017 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.162998 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:13:41.163111 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.163095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:13:41.167685 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.167664 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wnzzp\"" Jul 08 11:13:41.168185 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.168169 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-469dh\"" Jul 08 11:13:41.186437 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.186406 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d96df899f-2wt7h"] Jul 08 11:13:41.187862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.187840 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 11:13:41.205400 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.205295 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh"] Jul 08 11:13:41.205400 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.205350 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb"] Jul 08 11:13:41.205400 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.205369 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9hns5"] Jul 08 11:13:41.226901 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.226874 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lkn52"] Jul 08 11:13:41.227863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.227839 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf"] Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-trusted-ca\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237317 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mwcg\" (UniqueName: \"kubernetes.io/projected/056b73f6-ff1d-49f9-b96b-d17a2c513b0e-kube-api-access-4mwcg\") pod \"volume-data-source-validator-5d456fdbcd-8vjds\" (UID: \"056b73f6-ff1d-49f9-b96b-d17a2c513b0e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-config-volume\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0393330b-f585-4071-afa5-7740af485277-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh962\" (UniqueName: \"kubernetes.io/projected/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-kube-api-access-xh962\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8m2\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52p4m\" (UniqueName: \"kubernetes.io/projected/37ba2488-948f-43c4-b05e-332f112544b7-kube-api-access-52p4m\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237544 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237568 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237594 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-serving-cert\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237627 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237763 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmt4b\" (UniqueName: \"kubernetes.io/projected/a02b54ca-d226-4494-bc5f-c3a28d197408-kube-api-access-xmt4b\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237788 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff9e3339-68f5-446f-aa70-884564afc60e-config\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237810 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-config\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237833 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ldsh\" (UniqueName: \"kubernetes.io/projected/0393330b-f585-4071-afa5-7740af485277-kube-api-access-6ldsh\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237893 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237917 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9e3339-68f5-446f-aa70-884564afc60e-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c62768a-f097-4ddb-b07b-835fd693b3c5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-default-certificate\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.237993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rpkx\" (UniqueName: \"kubernetes.io/projected/ff9e3339-68f5-446f-aa70-884564afc60e-kube-api-access-4rpkx\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238019 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37ba2488-948f-43c4-b05e-332f112544b7-serving-cert\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238040 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238061 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-tmp-dir\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.238668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238087 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-stats-auth\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhvz6\" (UniqueName: \"kubernetes.io/projected/cd7229a4-1510-410a-8702-30ecb337e602-kube-api-access-hhvz6\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7lvj\" (UniqueName: \"kubernetes.io/projected/fbc898d5-8f90-406f-a2d0-bae27b06dff4-kube-api-access-q7lvj\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-tmp\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbc898d5-8f90-406f-a2d0-bae27b06dff4-cert\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238256 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0393330b-f585-4071-afa5-7740af485277-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238320 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-snapshots\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg5rn\" (UniqueName: \"kubernetes.io/projected/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-kube-api-access-fg5rn\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjtcb\" (UniqueName: \"kubernetes.io/projected/1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f-kube-api-access-fjtcb\") pod \"network-check-source-989f8b8dc-wm2sf\" (UID: \"1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwh2k\" (UniqueName: \"kubernetes.io/projected/7c62768a-f097-4ddb-b07b-835fd693b3c5-kube-api-access-hwh2k\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.239402 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.238567 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:41.240082 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.238581 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:41.240082 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.238646 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.738615987 +0000 UTC m=+38.120262735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:41.240082 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238963 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.240082 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.238978 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-snapshots\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.240263 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.240131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37ba2488-948f-43c4-b05e-332f112544b7-tmp\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.240890 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.240869 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.241411 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.241388 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37ba2488-948f-43c4-b05e-332f112544b7-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.241533 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.241415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0393330b-f585-4071-afa5-7740af485277-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.241602 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.241541 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.243828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.243805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0393330b-f585-4071-afa5-7740af485277-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.243934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.243911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.243934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.243917 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/37ba2488-948f-43c4-b05e-332f112544b7-serving-cert\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.243934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.243925 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.270003 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.269976 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d"] Jul 08 11:13:41.288063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.288033 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ldsh\" (UniqueName: \"kubernetes.io/projected/0393330b-f585-4071-afa5-7740af485277-kube-api-access-6ldsh\") pod \"kube-storage-version-migrator-operator-846bf668f7-85mhl\" (UID: \"0393330b-f585-4071-afa5-7740af485277\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.290600 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.290579 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx8m2\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.298376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.298350 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52p4m\" (UniqueName: \"kubernetes.io/projected/37ba2488-948f-43c4-b05e-332f112544b7-kube-api-access-52p4m\") pod \"insights-operator-5c847dcfc4-2zrxz\" (UID: \"37ba2488-948f-43c4-b05e-332f112544b7\") " pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.300522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.300500 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mwcg\" (UniqueName: \"kubernetes.io/projected/056b73f6-ff1d-49f9-b96b-d17a2c513b0e-kube-api-access-4mwcg\") pod \"volume-data-source-validator-5d456fdbcd-8vjds\" (UID: \"056b73f6-ff1d-49f9-b96b-d17a2c513b0e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" Jul 08 11:13:41.301589 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.301545 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.339103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.339103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjtcb\" (UniqueName: \"kubernetes.io/projected/1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f-kube-api-access-fjtcb\") pod \"network-check-source-989f8b8dc-wm2sf\" (UID: \"1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" Jul 08 11:13:41.339103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwh2k\" (UniqueName: \"kubernetes.io/projected/7c62768a-f097-4ddb-b07b-835fd693b3c5-kube-api-access-hwh2k\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-trusted-ca\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-config-volume\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh962\" (UniqueName: \"kubernetes.io/projected/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-kube-api-access-xh962\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339247 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-serving-cert\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmt4b\" (UniqueName: \"kubernetes.io/projected/a02b54ca-d226-4494-bc5f-c3a28d197408-kube-api-access-xmt4b\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.339360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339359 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff9e3339-68f5-446f-aa70-884564afc60e-config\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-config\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339466 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9e3339-68f5-446f-aa70-884564afc60e-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339491 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c62768a-f097-4ddb-b07b-835fd693b3c5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-default-certificate\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rpkx\" (UniqueName: \"kubernetes.io/projected/ff9e3339-68f5-446f-aa70-884564afc60e-kube-api-access-4rpkx\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-tmp-dir\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339620 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-stats-auth\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhvz6\" (UniqueName: \"kubernetes.io/projected/cd7229a4-1510-410a-8702-30ecb337e602-kube-api-access-hhvz6\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7lvj\" (UniqueName: \"kubernetes.io/projected/fbc898d5-8f90-406f-a2d0-bae27b06dff4-kube-api-access-q7lvj\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339750 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbc898d5-8f90-406f-a2d0-bae27b06dff4-cert\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.339780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339767 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.339847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg5rn\" (UniqueName: \"kubernetes.io/projected/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-kube-api-access-fg5rn\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.339874 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.339943 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.839926102 +0000 UTC m=+38.221572857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.340337 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:41.340420 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.340382 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.840368401 +0000 UTC m=+38.222015152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:41.341120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.341096 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c62768a-f097-4ddb-b07b-835fd693b3c5-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.342007 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.341985 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:41.342129 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.342031 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-trusted-ca\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.342129 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.342046 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.84202977 +0000 UTC m=+38.223676519 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:41.342241 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.342223 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-tmp-dir\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.342342 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.342320 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-config-volume\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.342342 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.342340 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.842324919 +0000 UTC m=+38.223971671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:41.342889 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.342585 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:41.342889 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.342668 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.842653185 +0000 UTC m=+38.224299955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:41.342889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.342834 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff9e3339-68f5-446f-aa70-884564afc60e-config\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.343142 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.342960 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:41.343142 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.343027 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:41.843005773 +0000 UTC m=+38.224652522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:41.343367 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.343347 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-config\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.344270 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.344243 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-default-certificate\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.344589 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.344561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-serving-cert\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.344687 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.344612 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff9e3339-68f5-446f-aa70-884564afc60e-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.345188 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.345163 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbc898d5-8f90-406f-a2d0-bae27b06dff4-cert\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.345515 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.345498 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-stats-auth\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.356357 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.355934 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" Jul 08 11:13:41.363247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.363224 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" Jul 08 11:13:41.365656 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.365617 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg5rn\" (UniqueName: \"kubernetes.io/projected/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-kube-api-access-fg5rn\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.379086 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.379060 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" Jul 08 11:13:41.380781 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.380760 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rpkx\" (UniqueName: \"kubernetes.io/projected/ff9e3339-68f5-446f-aa70-884564afc60e-kube-api-access-4rpkx\") pod \"service-ca-operator-5d46d9ffd4-j58xb\" (UID: \"ff9e3339-68f5-446f-aa70-884564afc60e\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.380865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.380801 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwh2k\" (UniqueName: \"kubernetes.io/projected/7c62768a-f097-4ddb-b07b-835fd693b3c5-kube-api-access-hwh2k\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.383813 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.383790 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7lvj\" (UniqueName: \"kubernetes.io/projected/fbc898d5-8f90-406f-a2d0-bae27b06dff4-kube-api-access-q7lvj\") pod \"ingress-canary-9hns5\" (UID: \"fbc898d5-8f90-406f-a2d0-bae27b06dff4\") " pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.383896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.383831 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjtcb\" (UniqueName: \"kubernetes.io/projected/1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f-kube-api-access-fjtcb\") pod \"network-check-source-989f8b8dc-wm2sf\" (UID: \"1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" Jul 08 11:13:41.384408 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.384385 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhvz6\" (UniqueName: \"kubernetes.io/projected/cd7229a4-1510-410a-8702-30ecb337e602-kube-api-access-hhvz6\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.385524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.385500 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh962\" (UniqueName: \"kubernetes.io/projected/a1e48b41-a320-49ad-89ed-6cd72d6f1bcb-kube-api-access-xh962\") pod \"console-operator-68b757fb67-zpsrs\" (UID: \"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb\") " pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.389529 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.389509 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmt4b\" (UniqueName: \"kubernetes.io/projected/a02b54ca-d226-4494-bc5f-c3a28d197408-kube-api-access-xmt4b\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.393679 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.393658 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:41.447348 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.447318 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" Jul 08 11:13:41.470136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.470105 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9hns5" Jul 08 11:13:41.476089 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.476066 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" Jul 08 11:13:41.537697 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.537662 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:41.753008 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.747741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:41.753008 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.747983 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:41.753008 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.748002 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:41.753008 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.748061 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.748040628 +0000 UTC m=+39.129687380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:41.765675 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.764558 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.851729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.852046 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.852119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.852178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.852206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:41.852244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.852388 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.852448 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.852428866 +0000 UTC m=+39.234075632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.852899 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.852955 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.852935698 +0000 UTC m=+39.234582451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853005 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853036 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.853025788 +0000 UTC m=+39.234672541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853082 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:41.853268 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853109 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.85310027 +0000 UTC m=+39.234747025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:41.854077 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853168 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.853158968 +0000 UTC m=+39.234805722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:41.854077 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853214 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:41.854077 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:41.853239 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:42.853230629 +0000 UTC m=+39.234877383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:42.020521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.020493 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9hns5"] Jul 08 11:13:42.026857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.026827 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds"] Jul 08 11:13:42.041863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.040436 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2zrxz"] Jul 08 11:13:42.041863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.041757 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb"] Jul 08 11:13:42.046910 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.046821 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl"] Jul 08 11:13:42.047785 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.047756 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf"] Jul 08 11:13:42.066325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.066303 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-zpsrs"] Jul 08 11:13:42.069284 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:42.069248 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ba2488_948f_43c4_b05e_332f112544b7.slice/crio-c159ea73c673c98c2b47c1836a415ef2dd5cd8ba3b9d5c39f4cbe90b312e40f8 WatchSource:0}: Error finding container c159ea73c673c98c2b47c1836a415ef2dd5cd8ba3b9d5c39f4cbe90b312e40f8: Status 404 returned error can't find the container with id c159ea73c673c98c2b47c1836a415ef2dd5cd8ba3b9d5c39f4cbe90b312e40f8 Jul 08 11:13:42.069967 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:42.069940 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e48b41_a320_49ad_89ed_6cd72d6f1bcb.slice/crio-ee6691c05f7a1f99d5b7f67e83be3747872ddf49c67da3fce4d0d00ea2242be5 WatchSource:0}: Error finding container ee6691c05f7a1f99d5b7f67e83be3747872ddf49c67da3fce4d0d00ea2242be5: Status 404 returned error can't find the container with id ee6691c05f7a1f99d5b7f67e83be3747872ddf49c67da3fce4d0d00ea2242be5 Jul 08 11:13:42.072390 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:42.071744 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcbe14e_2aa6_4c28_b2f0_3b620416ff3f.slice/crio-c9c59df9699d073a0eaca06fb91781e94ac7fd4f6098d91741f3aa4ea1a6a89b WatchSource:0}: Error finding container c9c59df9699d073a0eaca06fb91781e94ac7fd4f6098d91741f3aa4ea1a6a89b: Status 404 returned error can't find the container with id c9c59df9699d073a0eaca06fb91781e94ac7fd4f6098d91741f3aa4ea1a6a89b Jul 08 11:13:42.125443 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:42.125415 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056b73f6_ff1d_49f9_b96b_d17a2c513b0e.slice/crio-4af6a12e6c0ba75ec87b2035465bf0d97da41e88c0a921a005d65bceef22df3e WatchSource:0}: Error finding container 4af6a12e6c0ba75ec87b2035465bf0d97da41e88c0a921a005d65bceef22df3e: Status 404 returned error can't find the container with id 4af6a12e6c0ba75ec87b2035465bf0d97da41e88c0a921a005d65bceef22df3e Jul 08 11:13:42.163748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.163719 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:13:42.170103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.169930 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 11:13:42.345422 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.345330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" event={"ID":"1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f","Type":"ContainerStarted","Data":"c9c59df9699d073a0eaca06fb91781e94ac7fd4f6098d91741f3aa4ea1a6a89b"} Jul 08 11:13:42.346359 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.346336 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" event={"ID":"056b73f6-ff1d-49f9-b96b-d17a2c513b0e","Type":"ContainerStarted","Data":"4af6a12e6c0ba75ec87b2035465bf0d97da41e88c0a921a005d65bceef22df3e"} Jul 08 11:13:42.347605 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.347584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" event={"ID":"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb","Type":"ContainerStarted","Data":"ee6691c05f7a1f99d5b7f67e83be3747872ddf49c67da3fce4d0d00ea2242be5"} Jul 08 11:13:42.348568 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.348546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" event={"ID":"ff9e3339-68f5-446f-aa70-884564afc60e","Type":"ContainerStarted","Data":"de000495f46e907332179d49eb4b2205723b2332980122fd3ae57c3bbdb3a0b5"} Jul 08 11:13:42.349560 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.349540 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" event={"ID":"0393330b-f585-4071-afa5-7740af485277","Type":"ContainerStarted","Data":"66b258844e4dbd95e4a914549ecd2e93152320baa85e8ed61ebab5d3b6d00f71"} Jul 08 11:13:42.350433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.350414 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" event={"ID":"37ba2488-948f-43c4-b05e-332f112544b7","Type":"ContainerStarted","Data":"c159ea73c673c98c2b47c1836a415ef2dd5cd8ba3b9d5c39f4cbe90b312e40f8"} Jul 08 11:13:42.351385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.351366 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9hns5" event={"ID":"fbc898d5-8f90-406f-a2d0-bae27b06dff4","Type":"ContainerStarted","Data":"ad31537acdab3d06946434118dae96224efb5c7225cc5aed2f1e1a914ebd7bf5"} Jul 08 11:13:42.353839 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.353816 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="f0e5b67562c6f63ddbdb1f7e684901f4750ebf255469e9eb6df246406b817324" exitCode=0 Jul 08 11:13:42.353922 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.353856 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"f0e5b67562c6f63ddbdb1f7e684901f4750ebf255469e9eb6df246406b817324"} Jul 08 11:13:42.761817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.761769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:42.762044 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.762012 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:42.762044 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.762044 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:42.762179 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.762114 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.762091193 +0000 UTC m=+41.143737943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.862896 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.862985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.863031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.863061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.863098 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:42.863143 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863282 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863347 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.863327914 +0000 UTC m=+41.244974667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863797 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863853 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.863836189 +0000 UTC m=+41.245482943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863910 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.863947 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.863934407 +0000 UTC m=+41.245581163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.864009 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.864001422 +0000 UTC m=+41.245648185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:42.864220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.864064 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:42.865132 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.864093 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.864083366 +0000 UTC m=+41.245730121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:42.865132 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.864143 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:42.865132 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:42.864181 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:44.864171023 +0000 UTC m=+41.245817777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:43.049415 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:43.049186 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:43.216081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:43.215956 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:43.368754 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:43.364328 2559 generic.go:356] "Generic (PLEG): container finished" podID="d9943525-32a2-42a2-8126-23080f05a39e" containerID="42de7353a29f3389e46a29e14d703ca9382b0d73e8fef5991733e7607ab66085" exitCode=0 Jul 08 11:13:43.368754 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:43.364415 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerDied","Data":"42de7353a29f3389e46a29e14d703ca9382b0d73e8fef5991733e7607ab66085"} Jul 08 11:13:43.991090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:43.990930 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:44.121094 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.121061 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:44.375940 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.375096 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2dww" event={"ID":"d9943525-32a2-42a2-8126-23080f05a39e","Type":"ContainerStarted","Data":"a751b28c4674c425bedbc94a1d45c8aafd1a73f94e9f5ae07747193445ea40af"} Jul 08 11:13:44.426218 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.426150 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x2dww" podStartSLOduration=8.609944213 podStartE2EDuration="40.426134792s" totalImagesPullingTime="31.816190579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:44.4253488 +0000 UTC m=+40.806995572" watchObservedRunningTime="2026-07-08 11:13:44.426134792 +0000 UTC m=+40.807781564" Jul 08 11:13:44.677972 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.677874 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:44.783258 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.783084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:44.783410 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.783270 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:44.783410 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.783299 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:44.783410 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.783367 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.783348173 +0000 UTC m=+45.164994940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:44.811179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.811143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:44.884152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884114 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:44.884309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:44.884309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:44.884309 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884282 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:44.884309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884293 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884347 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.884327461 +0000 UTC m=+45.265974233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884372 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.884397 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884407 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884435 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.884417733 +0000 UTC m=+45.266064486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884455 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.88444573 +0000 UTC m=+45.266092484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884473 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.884461422 +0000 UTC m=+45.266108174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884475 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:44.884512 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884484 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:44.884902 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884526 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.884497653 +0000 UTC m=+45.266144424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:44.884902 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:44.884544 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:48.884534108 +0000 UTC m=+45.266180863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:44.943003 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:44.942890 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:45.930196 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:45.930160 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:47.798788 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:47.798753 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:47.929189 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:47.929143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:48.817400 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.817365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:48.817843 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.817499 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:48.817843 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.817511 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:48.817843 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.817584 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.817564744 +0000 UTC m=+53.199211494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:48.917779 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917872 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.917913 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:48.917959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:48.917955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.917990 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.917968294 +0000 UTC m=+53.299615064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918043 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918085 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.918073867 +0000 UTC m=+53.299720616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918125 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918153 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.918146414 +0000 UTC m=+53.299793163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918191 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:48.918242 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918217 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.918204739 +0000 UTC m=+53.299851488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:48.918495 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918261 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.918253522 +0000 UTC m=+53.299900270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:48.918495 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918298 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:48.918495 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:48.918319 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:56.918313633 +0000 UTC m=+53.299960381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:50.392932 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.391960 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" event={"ID":"1fcbe14e-2aa6-4c28-b2f0-3b620416ff3f","Type":"ContainerStarted","Data":"a1b5ab60ae3e348ddb8228278e0c9329d234db9acbfa27de54cfe4be4e35bdc9"} Jul 08 11:13:50.395565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.395277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" event={"ID":"056b73f6-ff1d-49f9-b96b-d17a2c513b0e","Type":"ContainerStarted","Data":"efa2b4eb9c56afdb336cc75d5882dc0643f0c7eaa6aa8979049117bc09abb973"} Jul 08 11:13:50.397153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.397123 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/0.log" Jul 08 11:13:50.397233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.397164 2559 generic.go:356] "Generic (PLEG): container finished" podID="a1e48b41-a320-49ad-89ed-6cd72d6f1bcb" containerID="afb22a981e21535d24bb12d7f14009ecd4b42764acb8bea1b55a450f93353ee3" exitCode=255 Jul 08 11:13:50.397233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.397228 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" event={"ID":"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb","Type":"ContainerDied","Data":"afb22a981e21535d24bb12d7f14009ecd4b42764acb8bea1b55a450f93353ee3"} Jul 08 11:13:50.397488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.397470 2559 scope.go:117] "RemoveContainer" containerID="afb22a981e21535d24bb12d7f14009ecd4b42764acb8bea1b55a450f93353ee3" Jul 08 11:13:50.401198 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.400854 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" event={"ID":"ff9e3339-68f5-446f-aa70-884564afc60e","Type":"ContainerStarted","Data":"f221dfc165b28015e837cee35591e94a06ac83657e04e6e9932eeb6189f68cf5"} Jul 08 11:13:50.402485 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.402463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" event={"ID":"0393330b-f585-4071-afa5-7740af485277","Type":"ContainerStarted","Data":"1a58ab8f75f494ef1be4b0ece51bfeaa4bdcd52e772d344eddca072555a5ce93"} Jul 08 11:13:50.404283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.404260 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" event={"ID":"37ba2488-948f-43c4-b05e-332f112544b7","Type":"ContainerStarted","Data":"f7e70bec510b92788e0ffa49b5b7c5a35a17130d2cf1558dfec06896a1d3d56b"} Jul 08 11:13:50.405803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.405781 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9hns5" event={"ID":"fbc898d5-8f90-406f-a2d0-bae27b06dff4","Type":"ContainerStarted","Data":"3930cd224e79aa92b733baa3f90eaea6f339ce1aaac3efad2147aca85ebbce9e"} Jul 08 11:13:50.423515 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.422695 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-wm2sf" podStartSLOduration=29.560838155 podStartE2EDuration="37.422678349s" totalImagesPullingTime="7.861840194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.421454832 +0000 UTC m=+46.803101601" watchObservedRunningTime="2026-07-08 11:13:50.422678349 +0000 UTC m=+46.804325123" Jul 08 11:13:50.518911 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.518855 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9hns5" podStartSLOduration=1.897672106 podStartE2EDuration="9.518832362s" totalImagesPullingTime="7.621160256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.483355517 +0000 UTC m=+46.865002289" watchObservedRunningTime="2026-07-08 11:13:50.518832362 +0000 UTC m=+46.900479134" Jul 08 11:13:50.554719 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.553863 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" podStartSLOduration=29.957475498 podStartE2EDuration="37.553840297s" totalImagesPullingTime="7.596364799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.552412926 +0000 UTC m=+46.934059698" watchObservedRunningTime="2026-07-08 11:13:50.553840297 +0000 UTC m=+46.935487069" Jul 08 11:13:50.554719 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.553978 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-8vjds" podStartSLOduration=30.066008178 podStartE2EDuration="37.55397272s" totalImagesPullingTime="7.487964542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.519326772 +0000 UTC m=+46.900973537" watchObservedRunningTime="2026-07-08 11:13:50.55397272 +0000 UTC m=+46.935619491" Jul 08 11:13:50.597890 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.597830 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" podStartSLOduration=29.986399332 podStartE2EDuration="37.597808355s" totalImagesPullingTime="7.611409023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.596817007 +0000 UTC m=+46.978463781" watchObservedRunningTime="2026-07-08 11:13:50.597808355 +0000 UTC m=+46.979455121" Jul 08 11:13:50.635673 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.635399 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" podStartSLOduration=29.020450971 podStartE2EDuration="36.635377176s" totalImagesPullingTime="7.614926205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:50.634520943 +0000 UTC m=+47.016167717" watchObservedRunningTime="2026-07-08 11:13:50.635377176 +0000 UTC m=+47.017023983" Jul 08 11:13:50.715424 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.715395 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:50.845026 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:50.844983 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:51.394439 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.394407 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:51.394823 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.394465 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:13:51.411554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.411527 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:13:51.411968 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.411953 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/0.log" Jul 08 11:13:51.412024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.411992 2559 generic.go:356] "Generic (PLEG): container finished" podID="a1e48b41-a320-49ad-89ed-6cd72d6f1bcb" containerID="cf28b38e425a57509596412caf142e7c811f389d6fc5dfe97de9e7ed981529fa" exitCode=255 Jul 08 11:13:51.412128 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.412101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" event={"ID":"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb","Type":"ContainerDied","Data":"cf28b38e425a57509596412caf142e7c811f389d6fc5dfe97de9e7ed981529fa"} Jul 08 11:13:51.412249 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.412167 2559 scope.go:117] "RemoveContainer" containerID="afb22a981e21535d24bb12d7f14009ecd4b42764acb8bea1b55a450f93353ee3" Jul 08 11:13:51.412760 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.412344 2559 scope.go:117] "RemoveContainer" containerID="cf28b38e425a57509596412caf142e7c811f389d6fc5dfe97de9e7ed981529fa" Jul 08 11:13:51.412760 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:51.412560 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-zpsrs_openshift-console-operator(a1e48b41-a320-49ad-89ed-6cd72d6f1bcb)\"" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" podUID="a1e48b41-a320-49ad-89ed-6cd72d6f1bcb" Jul 08 11:13:51.481220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.481186 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7qcs6"] Jul 08 11:13:51.518993 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.518961 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7qcs6"] Jul 08 11:13:51.519146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.519099 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.526962 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.526932 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 11:13:51.527083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.526956 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 11:13:51.527161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.527114 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5prrl\"" Jul 08 11:13:51.544875 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.544842 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.545025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.544965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zm77\" (UniqueName: \"kubernetes.io/projected/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-api-access-4zm77\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.545132 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.545115 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-crio-socket\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.545167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.545146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.545212 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.545174 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-data-volume\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.549257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.549230 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:51.645907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.645819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-crio-socket\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.645907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.645868 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.645907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.645897 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-data-volume\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.646161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.645973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.646161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.646042 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zm77\" (UniqueName: \"kubernetes.io/projected/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-api-access-4zm77\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.646161 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:51.646046 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 11:13:51.646161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.646060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-crio-socket\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.646161 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:51.646154 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 11:13:51.646346 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:51.646216 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:52.146195295 +0000 UTC m=+48.527842051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : configmap "kube-rbac-proxy" not found Jul 08 11:13:51.646346 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:51.646247 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:52.146236168 +0000 UTC m=+48.527882917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : secret "insights-runtime-extractor-tls" not found Jul 08 11:13:51.646346 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.646272 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-data-volume\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.671614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.671586 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zm77\" (UniqueName: \"kubernetes.io/projected/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-api-access-4zm77\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:51.769376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.769340 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:51.915383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:51.915294 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.065509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.065469 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.151310 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.151270 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:52.151495 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.151341 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:52.151495 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:52.151412 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 11:13:52.151600 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:52.151509 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 11:13:52.151600 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:52.151514 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:53.151491474 +0000 UTC m=+49.533138238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : configmap "kube-rbac-proxy" not found Jul 08 11:13:52.151600 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:52.151586 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:53.151571172 +0000 UTC m=+49.533217921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : secret "insights-runtime-extractor-tls" not found Jul 08 11:13:52.185876 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.185805 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.314353 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.314319 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.419346 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.419321 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:13:52.419867 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.419649 2559 scope.go:117] "RemoveContainer" containerID="cf28b38e425a57509596412caf142e7c811f389d6fc5dfe97de9e7ed981529fa" Jul 08 11:13:52.419867 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:52.419803 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-zpsrs_openshift-console-operator(a1e48b41-a320-49ad-89ed-6cd72d6f1bcb)\"" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" podUID="a1e48b41-a320-49ad-89ed-6cd72d6f1bcb" Jul 08 11:13:52.434889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.434860 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.461230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.461209 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2hf4f_ac3d9ce1-d290-4f67-80d2-98e05b218cc2/dns-node-resolver/0.log" Jul 08 11:13:52.680824 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.680791 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.812089 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.812009 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:52.984720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.984686 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8"] Jul 08 11:13:52.996546 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:52.996516 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" Jul 08 11:13:53.003083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.003057 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 11:13:53.004040 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.004014 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 11:13:53.004157 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.004117 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qj6v4\"" Jul 08 11:13:53.005669 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.005647 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8"] Jul 08 11:13:53.033242 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.033214 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:53.060378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.060339 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxp6\" (UniqueName: \"kubernetes.io/projected/61dfaeb2-3703-42fd-a32a-cb2f495a564f-kube-api-access-4sxp6\") pod \"migrator-d57d58cc9-vfkb8\" (UID: \"61dfaeb2-3703-42fd-a32a-cb2f495a564f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" Jul 08 11:13:53.138811 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.138707 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:53.160964 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.160927 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:53.161140 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.160982 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxp6\" (UniqueName: \"kubernetes.io/projected/61dfaeb2-3703-42fd-a32a-cb2f495a564f-kube-api-access-4sxp6\") pod \"migrator-d57d58cc9-vfkb8\" (UID: \"61dfaeb2-3703-42fd-a32a-cb2f495a564f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" Jul 08 11:13:53.161140 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:53.161054 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 11:13:53.161140 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:53.161129 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:55.161111172 +0000 UTC m=+51.542757924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : configmap "kube-rbac-proxy" not found Jul 08 11:13:53.161306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.161176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:53.161345 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:53.161315 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 11:13:53.161397 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:53.161371 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:55.161357548 +0000 UTC m=+51.543004297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : secret "insights-runtime-extractor-tls" not found Jul 08 11:13:53.170456 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.170422 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxp6\" (UniqueName: \"kubernetes.io/projected/61dfaeb2-3703-42fd-a32a-cb2f495a564f-kube-api-access-4sxp6\") pod \"migrator-d57d58cc9-vfkb8\" (UID: \"61dfaeb2-3703-42fd-a32a-cb2f495a564f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" Jul 08 11:13:53.236663 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.236596 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:53.306116 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.306073 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" Jul 08 11:13:53.332117 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.332075 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:53.447821 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.447796 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8"] Jul 08 11:13:53.450069 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:53.450043 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dfaeb2_3703_42fd_a32a_cb2f495a564f.slice/crio-2663ca96a38bf1aa028a1be811014f4e651bef7436107099be501f31e06584df WatchSource:0}: Error finding container 2663ca96a38bf1aa028a1be811014f4e651bef7436107099be501f31e06584df: Status 404 returned error can't find the container with id 2663ca96a38bf1aa028a1be811014f4e651bef7436107099be501f31e06584df Jul 08 11:13:53.656439 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.656359 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-blkb4_320a10ed-50a0-4a8e-8246-167e3a923848/node-ca/0.log" Jul 08 11:13:53.952461 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:53.952372 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:54.276144 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.276111 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9hns5_fbc898d5-8f90-406f-a2d0-bae27b06dff4/serve-healthcheck-canary/0.log" Jul 08 11:13:54.278682 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.278652 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-px5d6"] Jul 08 11:13:54.296586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.296550 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.306074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.306036 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zkmw5\"" Jul 08 11:13:54.306197 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.306116 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 11:13:54.306267 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.306039 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 11:13:54.306456 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.306440 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 11:13:54.307361 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.307335 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-px5d6"] Jul 08 11:13:54.308216 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.308195 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 11:13:54.340453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.340418 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:54.371966 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.371932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-key\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.372127 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.372014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbrc\" (UniqueName: \"kubernetes.io/projected/95607df2-d4af-47ab-88f1-47066f57dfa9-kube-api-access-bgbrc\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.372127 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.372069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-cabundle\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.427618 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.427585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" event={"ID":"61dfaeb2-3703-42fd-a32a-cb2f495a564f","Type":"ContainerStarted","Data":"2663ca96a38bf1aa028a1be811014f4e651bef7436107099be501f31e06584df"} Jul 08 11:13:54.472994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.472951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-key\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.473365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.473043 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbrc\" (UniqueName: \"kubernetes.io/projected/95607df2-d4af-47ab-88f1-47066f57dfa9-kube-api-access-bgbrc\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.473365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.473067 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-cabundle\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.473793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.473776 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-cabundle\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.475612 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.475588 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/95607df2-d4af-47ab-88f1-47066f57dfa9-signing-key\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.476243 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.476213 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:54.486872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.486848 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbrc\" (UniqueName: \"kubernetes.io/projected/95607df2-d4af-47ab-88f1-47066f57dfa9-kube-api-access-bgbrc\") pod \"service-ca-758f599c5-px5d6\" (UID: \"95607df2-d4af-47ab-88f1-47066f57dfa9\") " pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.608551 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.608467 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-px5d6" Jul 08 11:13:54.764981 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:54.764934 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-px5d6"] Jul 08 11:13:54.768442 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:13:54.768393 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95607df2_d4af_47ab_88f1_47066f57dfa9.slice/crio-88be65010110163e357dae637dfc4f4aec5230bacfcf58ffafb06b163fa46a29 WatchSource:0}: Error finding container 88be65010110163e357dae637dfc4f4aec5230bacfcf58ffafb06b163fa46a29: Status 404 returned error can't find the container with id 88be65010110163e357dae637dfc4f4aec5230bacfcf58ffafb06b163fa46a29 Jul 08 11:13:55.179356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.179272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:55.179510 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.179381 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:55.179589 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:55.179567 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 11:13:55.179683 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:55.179670 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:13:59.179647649 +0000 UTC m=+55.561294412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : secret "insights-runtime-extractor-tls" not found Jul 08 11:13:55.179875 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.179856 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:55.432511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.432425 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-px5d6" event={"ID":"95607df2-d4af-47ab-88f1-47066f57dfa9","Type":"ContainerStarted","Data":"f4ce2b0a4201a5862b53a5107c329ba86498f88ebe34b43c315463c13acc8610"} Jul 08 11:13:55.432511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.432481 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-px5d6" event={"ID":"95607df2-d4af-47ab-88f1-47066f57dfa9","Type":"ContainerStarted","Data":"88be65010110163e357dae637dfc4f4aec5230bacfcf58ffafb06b163fa46a29"} Jul 08 11:13:55.481150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.481096 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-px5d6" podStartSLOduration=1.481078821 podStartE2EDuration="1.481078821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:55.480400185 +0000 UTC m=+51.862046955" watchObservedRunningTime="2026-07-08 11:13:55.481078821 +0000 UTC m=+51.862725653" Jul 08 11:13:55.973122 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:55.973097 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.317715 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.317684 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.440486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.440444 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" event={"ID":"61dfaeb2-3703-42fd-a32a-cb2f495a564f","Type":"ContainerStarted","Data":"48cb0ad0884cb8b33f413a68229ca1dda9dfe6bb49329cd80e5a6780f1ee41b0"} Jul 08 11:13:56.440705 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.440527 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" event={"ID":"61dfaeb2-3703-42fd-a32a-cb2f495a564f","Type":"ContainerStarted","Data":"fe52de6feacfcb78f62ae35b2207db711b2dc6dfadcc74ee69ab36e14cea0033"} Jul 08 11:13:56.470450 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.470407 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.615249 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.615143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.774133 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.774093 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.896560 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.896462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:13:56.896763 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:56.896648 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 11:13:56.896763 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:56.896676 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6884ccd8f9-rfwgr: secret "image-registry-tls" not found Jul 08 11:13:56.896763 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:56.896750 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls podName:1e79a886-6964-48f8-b870-0a6ed3bd464a nodeName:}" failed. No retries permitted until 2026-07-08 11:14:12.896728811 +0000 UTC m=+69.278375574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls") pod "image-registry-6884ccd8f9-rfwgr" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a") : secret "image-registry-tls" not found Jul 08 11:13:56.914727 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.914696 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:56.997999 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.997958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:13:56.998212 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.998025 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:56.998501 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.998481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:13:56.998562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:56.998541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.000696 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:57.000783 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.000808 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert podName:d96103c4-bf70-4902-9d90-d70e6c6b2a9c nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.000786414 +0000 UTC m=+69.382433168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-m7sbh" (UID: "d96103c4-bf70-4902-9d90-d70e6c6b2a9c") : secret "networking-console-plugin-cert" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.000893 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:57.000915 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.000953 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls podName:cd7229a4-1510-410a-8702-30ecb337e602 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.000936581 +0000 UTC m=+69.382583333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-t2xnf" (UID: "cd7229a4-1510-410a-8702-30ecb337e602") : secret "samples-operator-tls" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001010 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001051 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls podName:7c62768a-f097-4ddb-b07b-835fd693b3c5 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.001039778 +0000 UTC m=+69.382686530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9l67d" (UID: "7c62768a-f097-4ddb-b07b-835fd693b3c5") : secret "cluster-monitoring-operator-tls" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001117 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.001100313 +0000 UTC m=+69.382747066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : configmap references non-existent config key: service-ca.crt Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001201 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 11:13:57.001278 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001238 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs podName:a02b54ca-d226-4494-bc5f-c3a28d197408 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.001221991 +0000 UTC m=+69.382868741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs") pod "router-default-5d96df899f-2wt7h" (UID: "a02b54ca-d226-4494-bc5f-c3a28d197408") : secret "router-metrics-certs-default" not found Jul 08 11:13:57.001899 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001481 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 11:13:57.001899 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:57.001526 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls podName:d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:13.001507155 +0000 UTC m=+69.383153906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls") pod "dns-default-lkn52" (UID: "d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37") : secret "dns-default-metrics-tls" not found Jul 08 11:13:57.095285 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:57.095249 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:57.191365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:57.191277 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:57.291904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:57.291872 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:58.041138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:58.041108 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:58.422696 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:58.422534 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:58.816222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:58.816195 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:58.917976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:58.917930 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:59.222846 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:59.222814 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:13:59.223259 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:59.222912 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:59.223259 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:59.222987 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 11:13:59.223259 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:13:59.223063 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls podName:95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2 nodeName:}" failed. No retries permitted until 2026-07-08 11:14:07.223047981 +0000 UTC m=+63.604694730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7qcs6" (UID: "95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2") : secret "insights-runtime-extractor-tls" not found Jul 08 11:13:59.616898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:59.616805 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:13:59.826369 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:13:59.826324 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.020083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.020040 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.355404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.355316 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.458124 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.458088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.649239 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.649148 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.784971 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.784938 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:00.899987 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:00.899908 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:01.030739 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.030669 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:01.120401 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.120364 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:01.394195 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.394164 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:14:01.394582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.394226 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:14:01.394582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.394573 2559 scope.go:117] "RemoveContainer" containerID="cf28b38e425a57509596412caf142e7c811f389d6fc5dfe97de9e7ed981529fa" Jul 08 11:14:01.626433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:01.626395 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.053511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.053475 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.153944 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.153911 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.253869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.253834 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.378671 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.378557 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.463329 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.463300 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:14:02.463815 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.463361 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" event={"ID":"a1e48b41-a320-49ad-89ed-6cd72d6f1bcb","Type":"ContainerStarted","Data":"73a103cadbd7223642d19a1c43dfe687f994e516a310ecdf707583470d590e40"} Jul 08 11:14:02.463815 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.463658 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:14:02.468379 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.468351 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" Jul 08 11:14:02.491042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.491016 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.497286 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.497241 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-zpsrs" podStartSLOduration=41.882795728 podStartE2EDuration="49.49722764s" totalImagesPullingTime="7.614431912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:02.496481041 +0000 UTC m=+58.878127811" watchObservedRunningTime="2026-07-08 11:14:02.49722764 +0000 UTC m=+58.878874409" Jul 08 11:14:02.498057 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.498025 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-vfkb8" podStartSLOduration=8.157380111 podStartE2EDuration="10.498016827s" totalImagesPullingTime="2.340636716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:13:56.483050197 +0000 UTC m=+52.864696970" watchObservedRunningTime="2026-07-08 11:14:02.498016827 +0000 UTC m=+58.879663576" Jul 08 11:14:02.587877 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.587840 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.816988 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.816950 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:02.967213 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:02.967152 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.066928 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.066894 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.170020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.169921 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.351942 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.351909 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.452519 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.452434 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.746137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.746104 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:03.930714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:03.930683 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:04.030844 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:04.030742 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:04.613915 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:04.613872 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:04.828470 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:04.828431 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:05.280849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:05.280808 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:05.454936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:05.454903 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:05.622818 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:05.622718 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:05.833332 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:05.833296 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:06.341219 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:06.341194 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fdzdr" Jul 08 11:14:06.433079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:06.433039 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:07.046932 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.046903 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:07.265425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.265389 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:07.292247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.292211 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:14:07.294553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.294535 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7qcs6\" (UID: \"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2\") " pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:14:07.432122 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.432037 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5prrl\"" Jul 08 11:14:07.440161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.440129 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7qcs6" Jul 08 11:14:07.468953 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.468915 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:07.659438 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.659407 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7qcs6"] Jul 08 11:14:07.669114 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:07.669074 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f9c2cf_9e8c_4601_aa7d_f3fa26e5e5e2.slice/crio-e7748b14139dcaeb5714750c69b80e3f29aa5189983101e49b0b2e85fa8519ce WatchSource:0}: Error finding container e7748b14139dcaeb5714750c69b80e3f29aa5189983101e49b0b2e85fa8519ce: Status 404 returned error can't find the container with id e7748b14139dcaeb5714750c69b80e3f29aa5189983101e49b0b2e85fa8519ce Jul 08 11:14:07.767716 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:07.767680 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:08.488663 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:08.488587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qcs6" event={"ID":"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2","Type":"ContainerStarted","Data":"e7748b14139dcaeb5714750c69b80e3f29aa5189983101e49b0b2e85fa8519ce"} Jul 08 11:14:09.816707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.816660 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:14:09.819692 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.819668 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 11:14:09.830113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.830082 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afb0dca8-0538-4ede-8a26-05a2d37df3a4-metrics-certs\") pod \"network-metrics-daemon-46kkp\" (UID: \"afb0dca8-0538-4ede-8a26-05a2d37df3a4\") " pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:14:09.918181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.918137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:14:09.918374 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.918210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:14:09.921227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.921193 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 11:14:09.921362 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.921271 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fzrh\" (UniqueName: \"kubernetes.io/projected/4d73a979-8fb1-47e6-93ef-bacda432632c-kube-api-access-4fzrh\") pod \"network-check-target-xlvnq\" (UID: \"4d73a979-8fb1-47e6-93ef-bacda432632c\") " pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:14:09.931283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.931252 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b358f565-efb3-4820-b6df-d5aed770cbb2-original-pull-secret\") pod \"global-pull-secret-syncer-w5f7p\" (UID: \"b358f565-efb3-4820-b6df-d5aed770cbb2\") " pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:14:09.987014 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.986984 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-469dh\"" Jul 08 11:14:09.993323 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.993298 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wnzzp\"" Jul 08 11:14:09.994313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:09.994293 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:14:10.000916 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:10.000890 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46kkp" Jul 08 11:14:10.073201 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:10.073127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w5f7p" Jul 08 11:14:11.457192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.457155 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w5f7p"] Jul 08 11:14:11.460421 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:11.460386 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb358f565_efb3_4820_b6df_d5aed770cbb2.slice/crio-33405c458ec161e3e90fb398c166656e8b0cd23198b23ab9e418c31e9ba86d4c WatchSource:0}: Error finding container 33405c458ec161e3e90fb398c166656e8b0cd23198b23ab9e418c31e9ba86d4c: Status 404 returned error can't find the container with id 33405c458ec161e3e90fb398c166656e8b0cd23198b23ab9e418c31e9ba86d4c Jul 08 11:14:11.468269 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.468230 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlvnq"] Jul 08 11:14:11.470664 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:11.470617 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d73a979_8fb1_47e6_93ef_bacda432632c.slice/crio-9f8d670df7e1de552af06e7a95c3a07ffea720c2f5eb8981339fe39608a9b2c7 WatchSource:0}: Error finding container 9f8d670df7e1de552af06e7a95c3a07ffea720c2f5eb8981339fe39608a9b2c7: Status 404 returned error can't find the container with id 9f8d670df7e1de552af06e7a95c3a07ffea720c2f5eb8981339fe39608a9b2c7 Jul 08 11:14:11.500409 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.500376 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w5f7p" event={"ID":"b358f565-efb3-4820-b6df-d5aed770cbb2","Type":"ContainerStarted","Data":"33405c458ec161e3e90fb398c166656e8b0cd23198b23ab9e418c31e9ba86d4c"} Jul 08 11:14:11.501480 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.501457 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlvnq" event={"ID":"4d73a979-8fb1-47e6-93ef-bacda432632c","Type":"ContainerStarted","Data":"9f8d670df7e1de552af06e7a95c3a07ffea720c2f5eb8981339fe39608a9b2c7"} Jul 08 11:14:11.502823 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.502802 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qcs6" event={"ID":"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2","Type":"ContainerStarted","Data":"c6e432138d33fbe640c585d840640a75632906513bca7b9906814918356f41f2"} Jul 08 11:14:11.534874 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.534805 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:11.702809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.702693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46kkp"] Jul 08 11:14:11.705223 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:11.705196 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb0dca8_0538_4ede_8a26_05a2d37df3a4.slice/crio-e85e105d37b78e5f44de60bdb8cfe30e53508861b9c29403741cf4b60b9e2eb6 WatchSource:0}: Error finding container e85e105d37b78e5f44de60bdb8cfe30e53508861b9c29403741cf4b60b9e2eb6: Status 404 returned error can't find the container with id e85e105d37b78e5f44de60bdb8cfe30e53508861b9c29403741cf4b60b9e2eb6 Jul 08 11:14:11.765018 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:11.764983 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:12.508597 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.508562 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qcs6" event={"ID":"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2","Type":"ContainerStarted","Data":"9eb33b237393e9a62cff7dd51c2d4f091f8460a4b01d426f2ec73eb9e433aae6"} Jul 08 11:14:12.509915 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.509884 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46kkp" event={"ID":"afb0dca8-0538-4ede-8a26-05a2d37df3a4","Type":"ContainerStarted","Data":"e85e105d37b78e5f44de60bdb8cfe30e53508861b9c29403741cf4b60b9e2eb6"} Jul 08 11:14:12.511544 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.511518 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlvnq" event={"ID":"4d73a979-8fb1-47e6-93ef-bacda432632c","Type":"ContainerStarted","Data":"bb93b2ae230884ebbcb8cd3139321f2d6fca894acca7f4c9ab9b37c547a2ee8c"} Jul 08 11:14:12.511686 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.511666 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:14:12.958345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.958301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:14:12.962409 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:12.962373 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"image-registry-6884ccd8f9-rfwgr\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:14:13.059047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:14:13.059222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059074 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:14:13.059222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:13.059222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:13.059222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:14:13.059222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.059184 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:14:13.060109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.060080 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a02b54ca-d226-4494-bc5f-c3a28d197408-service-ca-bundle\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:13.062262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.062234 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37-metrics-tls\") pod \"dns-default-lkn52\" (UID: \"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37\") " pod="openshift-dns/dns-default-lkn52" Jul 08 11:14:13.062363 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.062258 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d96103c4-bf70-4902-9d90-d70e6c6b2a9c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-m7sbh\" (UID: \"d96103c4-bf70-4902-9d90-d70e6c6b2a9c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:14:13.062363 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.062285 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a02b54ca-d226-4494-bc5f-c3a28d197408-metrics-certs\") pod \"router-default-5d96df899f-2wt7h\" (UID: \"a02b54ca-d226-4494-bc5f-c3a28d197408\") " pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:13.062363 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.062319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cd7229a4-1510-410a-8702-30ecb337e602-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-t2xnf\" (UID: \"cd7229a4-1510-410a-8702-30ecb337e602\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:14:13.062727 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.062705 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c62768a-f097-4ddb-b07b-835fd693b3c5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9l67d\" (UID: \"7c62768a-f097-4ddb-b07b-835fd693b3c5\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:14:13.176527 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.176496 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4bbvl\"" Jul 08 11:14:13.181235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.181201 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:14:13.190405 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.190357 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-d9598\"" Jul 08 11:14:13.197114 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.197086 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" Jul 08 11:14:13.238200 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.237970 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cp6jk\"" Jul 08 11:14:13.244196 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.244023 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p8r5m\"" Jul 08 11:14:13.244590 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.244547 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lkn52" Jul 08 11:14:13.249764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.249736 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:13.261759 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.259435 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r7795\"" Jul 08 11:14:13.269189 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.265648 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" Jul 08 11:14:13.269189 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.267815 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rmg2f\"" Jul 08 11:14:13.272491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.272465 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" Jul 08 11:14:13.431448 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.431387 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xlvnq" podStartSLOduration=69.431368767 podStartE2EDuration="1m9.431368767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:12.558483416 +0000 UTC m=+68.940130198" watchObservedRunningTime="2026-07-08 11:14:13.431368767 +0000 UTC m=+69.813015537" Jul 08 11:14:13.436974 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.435079 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:14:13.446653 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.446604 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf"] Jul 08 11:14:13.494874 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.494794 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lkn52"] Jul 08 11:14:13.526852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:13.523784 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e79a886_6964_48f8_b870_0a6ed3bd464a.slice/crio-bf793d51c28958ac757afd9013ffe608e68ce451110d5c21322c0000e7fb651b WatchSource:0}: Error finding container bf793d51c28958ac757afd9013ffe608e68ce451110d5c21322c0000e7fb651b: Status 404 returned error can't find the container with id bf793d51c28958ac757afd9013ffe608e68ce451110d5c21322c0000e7fb651b Jul 08 11:14:13.526852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:13.526168 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48f25bc_6a76_49eb_b1ae_6fd38a8c0a37.slice/crio-66501def5e2abb4fa36ec3902fd7c0f8fa964d1ebbeae6306a3deca5cb4ee925 WatchSource:0}: Error finding container 66501def5e2abb4fa36ec3902fd7c0f8fa964d1ebbeae6306a3deca5cb4ee925: Status 404 returned error can't find the container with id 66501def5e2abb4fa36ec3902fd7c0f8fa964d1ebbeae6306a3deca5cb4ee925 Jul 08 11:14:13.792596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.792562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh"] Jul 08 11:14:13.867866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.867838 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d96df899f-2wt7h"] Jul 08 11:14:13.868791 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.868711 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d"] Jul 08 11:14:13.869390 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:13.869304 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02b54ca_d226_4494_bc5f_c3a28d197408.slice/crio-64df99eb8bd38b35b4f6dc8723a8b26dbfca387ef4efa6fe9d98d8d7f3946b22 WatchSource:0}: Error finding container 64df99eb8bd38b35b4f6dc8723a8b26dbfca387ef4efa6fe9d98d8d7f3946b22: Status 404 returned error can't find the container with id 64df99eb8bd38b35b4f6dc8723a8b26dbfca387ef4efa6fe9d98d8d7f3946b22 Jul 08 11:14:13.869680 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:13.869623 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c62768a_f097_4ddb_b07b_835fd693b3c5.slice/crio-d94640c928dcd41a37c535491db49bf4bb151b3a1af1de5d39c46cd0d11e9070 WatchSource:0}: Error finding container d94640c928dcd41a37c535491db49bf4bb151b3a1af1de5d39c46cd0d11e9070: Status 404 returned error can't find the container with id d94640c928dcd41a37c535491db49bf4bb151b3a1af1de5d39c46cd0d11e9070 Jul 08 11:14:13.995550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:13.995514 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:14.469608 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.469558 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:14.524773 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.524734 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46kkp" event={"ID":"afb0dca8-0538-4ede-8a26-05a2d37df3a4","Type":"ContainerStarted","Data":"32e4a965ab0fea9964d0f0eae6d00b4c692ca0045fc6170f93f27aca570bff3d"} Jul 08 11:14:14.524947 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.524803 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46kkp" event={"ID":"afb0dca8-0538-4ede-8a26-05a2d37df3a4","Type":"ContainerStarted","Data":"b5ff4d0c579437eb89570f03f3da9226475018eaa9c5a7dd3f85f0a8a0687429"} Jul 08 11:14:14.527966 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.527935 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" event={"ID":"1e79a886-6964-48f8-b870-0a6ed3bd464a","Type":"ContainerStarted","Data":"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a"} Jul 08 11:14:14.528355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.527990 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" event={"ID":"1e79a886-6964-48f8-b870-0a6ed3bd464a","Type":"ContainerStarted","Data":"bf793d51c28958ac757afd9013ffe608e68ce451110d5c21322c0000e7fb651b"} Jul 08 11:14:14.528512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.528494 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:14:14.531336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.531309 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkn52" event={"ID":"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37","Type":"ContainerStarted","Data":"66501def5e2abb4fa36ec3902fd7c0f8fa964d1ebbeae6306a3deca5cb4ee925"} Jul 08 11:14:14.533384 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.533358 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" event={"ID":"d96103c4-bf70-4902-9d90-d70e6c6b2a9c","Type":"ContainerStarted","Data":"fdd1cadcd31bbf520ecd50ac6a3e8c038f1597510956f74c2fed8845c065c0cc"} Jul 08 11:14:14.535362 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.535333 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d96df899f-2wt7h" event={"ID":"a02b54ca-d226-4494-bc5f-c3a28d197408","Type":"ContainerStarted","Data":"0e4a33fc679d46ef973499fcee3dc84ed784fce59ab2b13425ad3e8b86095430"} Jul 08 11:14:14.535464 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.535381 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d96df899f-2wt7h" event={"ID":"a02b54ca-d226-4494-bc5f-c3a28d197408","Type":"ContainerStarted","Data":"64df99eb8bd38b35b4f6dc8723a8b26dbfca387ef4efa6fe9d98d8d7f3946b22"} Jul 08 11:14:14.537294 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.537259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" event={"ID":"7c62768a-f097-4ddb-b07b-835fd693b3c5","Type":"ContainerStarted","Data":"d94640c928dcd41a37c535491db49bf4bb151b3a1af1de5d39c46cd0d11e9070"} Jul 08 11:14:14.538731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.538708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" event={"ID":"cd7229a4-1510-410a-8702-30ecb337e602","Type":"ContainerStarted","Data":"58f1a76788810b356ba9f7345e3f270bde7129655fe89f203ea45640f4c92009"} Jul 08 11:14:14.583199 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.583145 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-46kkp" podStartSLOduration=68.724016438 podStartE2EDuration="1m10.58312497s" totalImagesPullingTime="1.859108532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:14.582547476 +0000 UTC m=+70.964194241" watchObservedRunningTime="2026-07-08 11:14:14.58312497 +0000 UTC m=+70.964771742" Jul 08 11:14:14.643258 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.643196 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" podStartSLOduration=70.643175138 podStartE2EDuration="1m10.643175138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:14.642237724 +0000 UTC m=+71.023884494" watchObservedRunningTime="2026-07-08 11:14:14.643175138 +0000 UTC m=+71.024821912" Jul 08 11:14:14.700770 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:14.700710 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podStartSLOduration=61.700687302 podStartE2EDuration="1m1.700687302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:14.700144931 +0000 UTC m=+71.081791702" watchObservedRunningTime="2026-07-08 11:14:14.700687302 +0000 UTC m=+71.082334072" Jul 08 11:14:15.251030 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:15.250996 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:15.254221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:15.254186 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:15.254221 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:15.254221 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:15.254221 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:15.254448 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:15.254247 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:16.252771 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:16.252732 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:16.252771 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:16.252771 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:16.252771 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:16.253282 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:16.252816 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:17.251827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:17.251784 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:17.251827 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:17.251827 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:17.251827 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:17.252201 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:17.251849 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:18.251601 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:18.251550 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:18.251601 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:18.251601 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:18.251601 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:18.252111 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:18.251626 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:18.515906 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:18.515819 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:18.719081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:18.719048 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:18.954626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:18.954589 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:19.223136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.223101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:19.251262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.251230 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:19.251262 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:19.251262 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:19.251262 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:19.251476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.251287 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:19.533849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.533748 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:19.568488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.568443 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" event={"ID":"d96103c4-bf70-4902-9d90-d70e6c6b2a9c","Type":"ContainerStarted","Data":"1851a016ec5493c769edb10a1cae71cf3b075ca3c9b75c040492e0ed0fd0ea03"} Jul 08 11:14:19.570185 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.570160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" event={"ID":"7c62768a-f097-4ddb-b07b-835fd693b3c5","Type":"ContainerStarted","Data":"430578b022e85e7672504181108dd65c2caac7ee2f68ce02d541e3ada1adb3db"} Jul 08 11:14:19.571830 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.571802 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" event={"ID":"cd7229a4-1510-410a-8702-30ecb337e602","Type":"ContainerStarted","Data":"4efa70d72b62178867d23d835428043184ea24d2726f4e2627c1a6335a3c2c8c"} Jul 08 11:14:19.571934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.571849 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" event={"ID":"cd7229a4-1510-410a-8702-30ecb337e602","Type":"ContainerStarted","Data":"e68a3a5a8e3bfa9d3478c8bc751c944e777cd4a9c7ebdb93e255f7726002c067"} Jul 08 11:14:19.573699 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.573666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7qcs6" event={"ID":"95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2","Type":"ContainerStarted","Data":"13e212c27d3b655e46dae61041a0f6cf39552f21b4857800f4378305491dbf32"} Jul 08 11:14:19.575450 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.575430 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w5f7p" event={"ID":"b358f565-efb3-4820-b6df-d5aed770cbb2","Type":"ContainerStarted","Data":"be297308cefbb2908d218752d3b7b3872f3b8bbf2b00457b65856e9653bb2f93"} Jul 08 11:14:19.577132 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.577110 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkn52" event={"ID":"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37","Type":"ContainerStarted","Data":"d10e7a7f1368a7b246a693a579c70cf123cb13f9bec5fd16593a592cca6ee767"} Jul 08 11:14:19.577205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.577152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkn52" event={"ID":"d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37","Type":"ContainerStarted","Data":"6f0ff0ebe4063c221f58deca21dde8039df1eee7a0e55a4ac435825329ac00b3"} Jul 08 11:14:19.577263 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.577246 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lkn52" Jul 08 11:14:19.636782 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.636726 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w5f7p" podStartSLOduration=68.353764025 podStartE2EDuration="1m15.63670884s" totalImagesPullingTime="7.282944815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.635550336 +0000 UTC m=+76.017197102" watchObservedRunningTime="2026-07-08 11:14:19.63670884 +0000 UTC m=+76.018355610" Jul 08 11:14:19.637170 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.637148 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-m7sbh" podStartSLOduration=60.688662832 podStartE2EDuration="1m5.637140962s" totalImagesPullingTime="4.94847813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.595417174 +0000 UTC m=+75.977063968" watchObservedRunningTime="2026-07-08 11:14:19.637140962 +0000 UTC m=+76.018787731" Jul 08 11:14:19.674598 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.674540 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-t2xnf" podStartSLOduration=61.570418736 podStartE2EDuration="1m6.674524347s" totalImagesPullingTime="5.104105611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.672872581 +0000 UTC m=+76.054519350" watchObservedRunningTime="2026-07-08 11:14:19.674524347 +0000 UTC m=+76.056171141" Jul 08 11:14:19.729509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.729449 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lkn52" podStartSLOduration=33.538367202 podStartE2EDuration="38.729428383s" totalImagesPullingTime="5.191061181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.726918957 +0000 UTC m=+76.108565728" watchObservedRunningTime="2026-07-08 11:14:19.729428383 +0000 UTC m=+76.111075155" Jul 08 11:14:19.754180 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.754146 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:19.764475 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.764423 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9l67d" podStartSLOduration=60.887382567 podStartE2EDuration="1m5.764401299s" totalImagesPullingTime="4.877018732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.763795288 +0000 UTC m=+76.145442059" watchObservedRunningTime="2026-07-08 11:14:19.764401299 +0000 UTC m=+76.146048070" Jul 08 11:14:19.826602 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.826484 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7qcs6" podStartSLOduration=19.135805729 podStartE2EDuration="28.826463549s" totalImagesPullingTime="9.69065782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:13:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:19.825533197 +0000 UTC m=+76.207179968" watchObservedRunningTime="2026-07-08 11:14:19.826463549 +0000 UTC m=+76.208110318" Jul 08 11:14:19.986615 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:19.986576 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:20.116586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:20.116496 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:20.251494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:20.251453 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:20.251494 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:20.251494 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:20.251494 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:20.251745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:20.251540 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:21.251251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:21.251217 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:21.251251 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:21.251251 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:21.251251 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:21.251707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:21.251288 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:22.251697 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:22.251661 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:22.251697 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:22.251697 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:22.251697 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:22.252111 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:22.251736 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:23.250038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:23.250002 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:14:23.251102 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:23.251074 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:23.251102 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:23.251102 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:23.251102 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:23.251290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:23.251135 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:24.214081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:24.214031 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:24.251851 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:24.251824 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:24.251851 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:24.251851 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:24.251851 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:24.252038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:24.251883 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:25.251123 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:25.251011 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:25.251123 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:25.251123 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:25.251123 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:25.251123 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:25.251063 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:26.251260 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:26.251218 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:26.251260 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:26.251260 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:26.251260 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:26.251720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:26.251285 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:26.867166 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:26.867107 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:27.007022 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:27.006991 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:27.251695 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:27.251653 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:27.251695 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:27.251695 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:27.251695 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:27.252129 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:27.251719 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:28.251432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:28.251394 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:28.251432 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:28.251432 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:28.251432 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:28.251723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:28.251451 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:29.251813 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:29.251777 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:29.251813 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:29.251813 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:29.251813 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:29.252258 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:29.251834 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:29.583379 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:29.583304 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lkn52" Jul 08 11:14:30.251038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:30.251008 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:30.251038 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:30.251038 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:30.251038 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:30.251276 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:30.251063 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:31.251528 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:31.251489 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:31.251528 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:31.251528 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:31.251528 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:31.251989 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:31.251545 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:32.251516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:32.251482 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:32.251516 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:32.251516 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:32.251516 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:32.251983 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:32.251538 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:33.185796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:33.185765 2559 patch_prober.go:28] interesting pod/image-registry-6884ccd8f9-rfwgr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 11:14:33.185972 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:33.185816 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 11:14:33.251257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:33.251223 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:33.251257 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:33.251257 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:33.251257 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:33.251476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:33.251278 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:34.251822 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:34.251784 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:34.251822 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:34.251822 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:34.251822 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:34.252312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:34.251853 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:35.251855 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:35.251819 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:35.251855 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:35.251855 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:35.251855 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:35.252333 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:35.251886 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:36.251170 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:36.251134 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:36.251170 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:36.251170 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:36.251170 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:36.251496 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:36.251210 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:36.552402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:36.552324 2559 patch_prober.go:28] interesting pod/image-registry-6884ccd8f9-rfwgr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 11:14:36.552402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:36.552374 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 11:14:37.251239 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:37.251206 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:37.251239 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:37.251239 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:37.251239 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:37.251470 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:37.251261 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:38.252020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:38.251987 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:38.252020 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:38.252020 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:38.252020 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:38.252471 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:38.252042 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:38.968251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:38.968210 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:39.080743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:39.080713 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:39.251900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:39.251822 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:39.251900 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:39.251900 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:39.251900 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:39.251900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:39.251871 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:40.191711 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.191678 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:40.251023 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.250988 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:40.251023 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:40.251023 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:40.251023 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:40.251337 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.251054 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:40.288317 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.288281 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:40.397700 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.397665 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:40.574358 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.574326 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:40.732279 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:40.732240 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:41.251554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:41.251514 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:41.251554 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:41.251554 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:41.251554 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:41.251796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:41.251574 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:41.297377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:41.297343 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:41.396336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:41.396301 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:42.251347 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:42.251309 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:42.251347 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:42.251347 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:42.251347 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:42.251583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:42.251372 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:43.186142 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.186107 2559 patch_prober.go:28] interesting pod/image-registry-6884ccd8f9-rfwgr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 11:14:43.186535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.186174 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 11:14:43.250956 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.250923 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:43.250956 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:43.250956 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:43.250956 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:43.251228 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.250980 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:43.519404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.519379 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xlvnq" Jul 08 11:14:43.908080 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.907975 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:43.980300 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:43.980257 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:44.251993 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:44.251959 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:44.251993 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:44.251993 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:44.251993 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:44.252507 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:44.252035 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:44.417196 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:44.417155 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:45.251005 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:45.250975 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:45.251005 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:45.251005 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:45.251005 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:45.251247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:45.251026 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:46.251458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:46.251421 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:46.251458 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:46.251458 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:46.251458 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:46.251932 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:46.251475 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:46.551822 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:46.551744 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:14:46.879401 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:46.879306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:47.044540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:47.044504 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:47.251172 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:47.251142 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:47.251172 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:47.251172 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:47.251172 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:47.251412 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:47.251206 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:47.256096 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:47.256072 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:48.251873 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:48.251834 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:48.251873 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:48.251873 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:48.251873 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:48.252113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:48.251894 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:49.195365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.195323 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:49.251673 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.251615 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:49.251673 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:49.251673 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:49.251673 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:49.251913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.251693 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:49.378133 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.378088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:49.779299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.779263 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:49.937589 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:49.937554 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5kcf5"] Jul 08 11:14:50.031016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.030922 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:50.165320 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.165288 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.172459 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.172434 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 11:14:50.173536 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.173510 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 11:14:50.173669 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.173518 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 11:14:50.173669 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.173615 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 11:14:50.173802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.173696 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bm97j\"" Jul 08 11:14:50.238113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.238076 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:50.247156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-tls\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247167 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-wtmp\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-sys\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-accelerators-collector-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-metrics-client-ca\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-root\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247443 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-textfile\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.247656 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.247497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgbf\" (UniqueName: \"kubernetes.io/projected/234e8778-b528-4487-90ca-1d3de9d6ab10-kube-api-access-9jgbf\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.251371 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.251337 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:50.251371 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:50.251371 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:50.251371 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:50.251587 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.251404 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:50.348230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348126 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-wtmp\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-sys\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348256 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-accelerators-collector-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348286 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-metrics-client-ca\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348310 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-root\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-textfile\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348353 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-sys\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348406 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-root\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348406 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-wtmp\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.348514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgbf\" (UniqueName: \"kubernetes.io/projected/234e8778-b528-4487-90ca-1d3de9d6ab10-kube-api-access-9jgbf\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.349153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348686 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-textfile\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.349153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-tls\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.349153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.348989 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-metrics-client-ca\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.349153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.349054 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-accelerators-collector-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.351280 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.351259 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.351389 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.351290 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/234e8778-b528-4487-90ca-1d3de9d6ab10-node-exporter-tls\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.366786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.366751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgbf\" (UniqueName: \"kubernetes.io/projected/234e8778-b528-4487-90ca-1d3de9d6ab10-kube-api-access-9jgbf\") pod \"node-exporter-5kcf5\" (UID: \"234e8778-b528-4487-90ca-1d3de9d6ab10\") " pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.419315 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.419272 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:50.475742 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.475703 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5kcf5" Jul 08 11:14:50.515148 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:14:50.515115 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234e8778_b528_4487_90ca_1d3de9d6ab10.slice/crio-479f22de5d3c305a12b32597b3a8b53f51305f3a90bfa397f680d9e01c48ef87 WatchSource:0}: Error finding container 479f22de5d3c305a12b32597b3a8b53f51305f3a90bfa397f680d9e01c48ef87: Status 404 returned error can't find the container with id 479f22de5d3c305a12b32597b3a8b53f51305f3a90bfa397f680d9e01c48ef87 Jul 08 11:14:50.690577 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.690496 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5kcf5" event={"ID":"234e8778-b528-4487-90ca-1d3de9d6ab10","Type":"ContainerStarted","Data":"479f22de5d3c305a12b32597b3a8b53f51305f3a90bfa397f680d9e01c48ef87"} Jul 08 11:14:50.765594 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.765559 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:50.895430 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:50.895394 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:51.251981 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:51.251929 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:51.251981 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:51.251981 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:51.251981 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:51.252572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:51.252006 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:52.251276 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:52.251237 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:52.251276 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:52.251276 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:52.251276 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:52.251517 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:52.251312 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:52.699249 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:52.699146 2559 generic.go:356] "Generic (PLEG): container finished" podID="234e8778-b528-4487-90ca-1d3de9d6ab10" containerID="1fe941f2dce8c9c6283d21d0e16098ea1f9633025d0b2854a2daf64be63d13ab" exitCode=0 Jul 08 11:14:52.699717 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:52.699245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5kcf5" event={"ID":"234e8778-b528-4487-90ca-1d3de9d6ab10","Type":"ContainerDied","Data":"1fe941f2dce8c9c6283d21d0e16098ea1f9633025d0b2854a2daf64be63d13ab"} Jul 08 11:14:53.251710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:53.251585 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:53.251710 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:53.251710 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:53.251710 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:53.251710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:53.251680 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:53.704937 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:53.704910 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5kcf5" event={"ID":"234e8778-b528-4487-90ca-1d3de9d6ab10","Type":"ContainerStarted","Data":"d131041a43b1adbae473f98b20cbbe9814c1fb7ddb121e182a7874816a891657"} Jul 08 11:14:53.705306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:53.705002 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5kcf5" event={"ID":"234e8778-b528-4487-90ca-1d3de9d6ab10","Type":"ContainerStarted","Data":"3ce8d73585bbc5a53d6d4d665f4b500e9d5601c56c53d5355391b94efd55a3a3"} Jul 08 11:14:54.251255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:54.251222 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:54.251255 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:54.251255 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:54.251255 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:54.251582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:54.251279 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:54.343060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:54.343023 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:54.590749 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:54.590652 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:54.764255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:54.764216 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:55.251889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:55.251843 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:55.251889 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:55.251889 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:55.251889 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:55.252120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:55.251903 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:56.251178 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:56.251139 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:56.251178 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:56.251178 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:56.251178 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:56.251614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:56.251202 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:56.434613 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:56.434575 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:56.549262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:56.549180 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:57.251705 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:57.251668 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:57.251705 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:57.251705 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:57.251705 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:57.252143 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:57.251729 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:57.491458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:57.491422 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:57.609727 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:57.609617 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:14:58.251599 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:58.251565 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:58.251599 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:58.251599 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:58.251599 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:58.252104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:58.251654 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:59.251737 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:59.251702 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:14:59.251737 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:14:59.251737 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:14:59.251737 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:14:59.252185 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:59.251777 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:14:59.872322 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:14:59.872291 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:00.013534 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.013493 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:00.111348 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.111309 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:00.216157 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.216103 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:00.251295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.251255 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:00.251295 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:00.251295 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:00.251295 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:00.251528 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.251336 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:00.385901 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:00.385866 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:01.251079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.251051 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:01.251079 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:01.251079 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:01.251079 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:01.251291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.251103 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:01.736847 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.736813 2559 generic.go:356] "Generic (PLEG): container finished" podID="ff9e3339-68f5-446f-aa70-884564afc60e" containerID="f221dfc165b28015e837cee35591e94a06ac83657e04e6e9932eeb6189f68cf5" exitCode=0 Jul 08 11:15:01.737195 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.736862 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" event={"ID":"ff9e3339-68f5-446f-aa70-884564afc60e","Type":"ContainerDied","Data":"f221dfc165b28015e837cee35591e94a06ac83657e04e6e9932eeb6189f68cf5"} Jul 08 11:15:01.737195 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.737141 2559 scope.go:117] "RemoveContainer" containerID="f221dfc165b28015e837cee35591e94a06ac83657e04e6e9932eeb6189f68cf5" Jul 08 11:15:01.770927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:01.770874 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5kcf5" podStartSLOduration=11.324596037 podStartE2EDuration="12.770857748s" totalImagesPullingTime="1.358176664s" totalInitContainerRuntime="88.085047ms" isStatefulPod=true podCreationTimestamp="2026-07-08 11:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:14:53.744683843 +0000 UTC m=+110.126330615" watchObservedRunningTime="2026-07-08 11:15:01.770857748 +0000 UTC m=+118.152504519" Jul 08 11:15:02.074250 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.074152 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:15:02.175299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.175265 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:02.251402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.251349 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:02.251402 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:02.251402 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:02.251402 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:02.251666 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.251420 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:02.284763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.284719 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:02.413346 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.413249 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:02.747500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:02.747460 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-j58xb" event={"ID":"ff9e3339-68f5-446f-aa70-884564afc60e","Type":"ContainerStarted","Data":"e365877ae86a8a8ee25d800a22f19bad559bc20a1d422cc15b8a981652217cec"} Jul 08 11:15:03.251179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:03.251145 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:03.251179 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:03.251179 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:03.251179 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:03.251410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:03.251205 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:04.251500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:04.251466 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:04.251500 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:04.251500 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:04.251500 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:04.251967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:04.251521 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:05.251852 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:05.251819 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:05.251852 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:05.251852 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:05.251852 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:05.252308 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:05.251873 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:06.251354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:06.251320 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:06.251354 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:06.251354 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:06.251354 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:06.251698 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:06.251380 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:07.251941 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:07.251907 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:07.251941 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:07.251941 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:07.251941 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:07.252386 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:07.251966 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:08.251112 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:08.251081 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:08.251112 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:08.251112 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:08.251112 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:08.251357 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:08.251132 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:09.251300 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:09.251256 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:09.251300 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:09.251300 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:09.251300 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:09.251812 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:09.251326 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:10.251504 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:10.251469 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:10.251504 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:10.251504 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:10.251504 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:10.251982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:10.251528 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:11.251583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:11.251546 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:11.251583 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:11.251583 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:11.251583 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:11.252037 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:11.251599 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:12.251485 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:12.251450 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:12.251485 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:12.251485 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:12.251485 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:12.251958 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:12.251507 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:13.251068 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:13.251033 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:13.251068 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:13.251068 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:13.251068 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:13.251314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:13.251088 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:14.251322 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:14.251285 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:14.251322 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:14.251322 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:14.251322 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:14.251803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:14.251352 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:15.250984 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:15.250945 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:15.250984 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:15.250984 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:15.250984 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:15.251312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:15.251019 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:16.251295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:16.251256 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:16.251295 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:16.251295 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:16.251295 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:16.251756 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:16.251317 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:17.251722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:17.251686 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:17.251722 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:17.251722 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:17.251722 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:17.252168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:17.251755 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:18.251119 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:18.251083 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:18.251119 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:18.251119 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:18.251119 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:18.251376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:18.251141 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:19.251763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:19.251727 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:19.251763 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:19.251763 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:19.251763 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:19.252186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:19.251783 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:20.251541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:20.251504 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:20.251541 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:20.251541 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:20.251541 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:20.252090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:20.251566 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:20.812720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:20.812684 2559 generic.go:356] "Generic (PLEG): container finished" podID="0393330b-f585-4071-afa5-7740af485277" containerID="1a58ab8f75f494ef1be4b0ece51bfeaa4bdcd52e772d344eddca072555a5ce93" exitCode=0 Jul 08 11:15:20.812898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:20.812761 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" event={"ID":"0393330b-f585-4071-afa5-7740af485277","Type":"ContainerDied","Data":"1a58ab8f75f494ef1be4b0ece51bfeaa4bdcd52e772d344eddca072555a5ce93"} Jul 08 11:15:20.813100 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:20.813085 2559 scope.go:117] "RemoveContainer" containerID="1a58ab8f75f494ef1be4b0ece51bfeaa4bdcd52e772d344eddca072555a5ce93" Jul 08 11:15:21.252027 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:21.251993 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:21.252027 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:21.252027 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:21.252027 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:21.252499 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:21.252062 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:21.713309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:21.713276 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:21.809934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:21.809902 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:21.818432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:21.818403 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-85mhl" event={"ID":"0393330b-f585-4071-afa5-7740af485277","Type":"ContainerStarted","Data":"af07d880f8a4df339049dbcbbee9ae22ae2e2393d606e224ffa0ec8b9219b394"} Jul 08 11:15:22.251268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:22.251237 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:22.251268 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:22.251268 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:22.251268 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:22.251500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:22.251296 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:23.250911 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:23.250873 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:23.250911 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:23.250911 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:23.250911 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:23.251340 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:23.250952 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:24.251123 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:24.251090 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:24.251123 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:24.251123 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:24.251123 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:24.251554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:24.251147 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:25.251496 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:25.251466 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:25.251496 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:25.251496 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:25.251496 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:25.251953 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:25.251516 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:25.835486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:25.835456 2559 generic.go:356] "Generic (PLEG): container finished" podID="37ba2488-948f-43c4-b05e-332f112544b7" containerID="f7e70bec510b92788e0ffa49b5b7c5a35a17130d2cf1558dfec06896a1d3d56b" exitCode=0 Jul 08 11:15:25.835667 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:25.835500 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" event={"ID":"37ba2488-948f-43c4-b05e-332f112544b7","Type":"ContainerDied","Data":"f7e70bec510b92788e0ffa49b5b7c5a35a17130d2cf1558dfec06896a1d3d56b"} Jul 08 11:15:25.835849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:25.835836 2559 scope.go:117] "RemoveContainer" containerID="f7e70bec510b92788e0ffa49b5b7c5a35a17130d2cf1558dfec06896a1d3d56b" Jul 08 11:15:26.251033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:26.251002 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:26.251033 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:26.251033 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:26.251033 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:26.251276 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:26.251056 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:26.306179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:26.306144 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:26.399819 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:26.399790 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:26.841293 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:26.841262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2zrxz" event={"ID":"37ba2488-948f-43c4-b05e-332f112544b7","Type":"ContainerStarted","Data":"18609b387edc2fa23dfbc7c0fde4b6841f1d3bfe760351704d3b6f72a8bdb4a2"} Jul 08 11:15:27.095902 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.095809 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" containerID="cri-o://7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a" gracePeriod=30 Jul 08 11:15:27.251679 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.251615 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:27.251679 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:27.251679 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:27.251679 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:27.251904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.251703 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:27.334528 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.334501 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:15:27.449617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449533 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449593 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449620 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449667 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449700 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449726 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx8m2\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449771 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.449841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.449807 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted\") pod \"1e79a886-6964-48f8-b870-0a6ed3bd464a\" (UID: \"1e79a886-6964-48f8-b870-0a6ed3bd464a\") " Jul 08 11:15:27.450101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.450019 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:15:27.450407 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.450350 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:15:27.453060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.453010 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:15:27.453060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.453019 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:15:27.453221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.453110 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2" (OuterVolumeSpecName: "kube-api-access-gx8m2") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "kube-api-access-gx8m2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:15:27.453221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.453132 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:15:27.453221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.453168 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:15:27.462173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.462139 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1e79a886-6964-48f8-b870-0a6ed3bd464a" (UID: "1e79a886-6964-48f8-b870-0a6ed3bd464a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:15:27.550977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.550943 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e79a886-6964-48f8-b870-0a6ed3bd464a-ca-trust-extracted\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.550977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.550972 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-trusted-ca\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.550977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.550982 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-installation-pull-secrets\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.551206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.550991 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-tls\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.551206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.551001 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e79a886-6964-48f8-b870-0a6ed3bd464a-image-registry-private-configuration\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.551206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.551010 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e79a886-6964-48f8-b870-0a6ed3bd464a-registry-certificates\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.551206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.551019 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx8m2\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-kube-api-access-gx8m2\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.551206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.551027 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e79a886-6964-48f8-b870-0a6ed3bd464a-bound-sa-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:15:27.846026 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.845996 2559 generic.go:356] "Generic (PLEG): container finished" podID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerID="7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a" exitCode=0 Jul 08 11:15:27.846209 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.846044 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" Jul 08 11:15:27.846209 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.846074 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" event={"ID":"1e79a886-6964-48f8-b870-0a6ed3bd464a","Type":"ContainerDied","Data":"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a"} Jul 08 11:15:27.846209 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.846112 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6884ccd8f9-rfwgr" event={"ID":"1e79a886-6964-48f8-b870-0a6ed3bd464a","Type":"ContainerDied","Data":"bf793d51c28958ac757afd9013ffe608e68ce451110d5c21322c0000e7fb651b"} Jul 08 11:15:27.846209 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.846131 2559 scope.go:117] "RemoveContainer" containerID="7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a" Jul 08 11:15:27.854872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.854856 2559 scope.go:117] "RemoveContainer" containerID="7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a" Jul 08 11:15:27.855132 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:15:27.855114 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a\": container with ID starting with 7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a not found: ID does not exist" containerID="7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a" Jul 08 11:15:27.855175 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.855140 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a"} err="failed to get container status \"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a\": rpc error: code = NotFound desc = could not find container \"7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a\": container with ID starting with 7d2119453416a2f38ea65cb31a5ed02728973d4ddcb22e84c9a074ac1e2ba24a not found: ID does not exist" Jul 08 11:15:27.877341 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.877306 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:15:27.889205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.889168 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6884ccd8f9-rfwgr"] Jul 08 11:15:27.994375 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:27.994335 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:28.168450 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:28.168374 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" path="/var/lib/kubelet/pods/1e79a886-6964-48f8-b870-0a6ed3bd464a/volumes" Jul 08 11:15:28.251997 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:28.251962 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:28.251997 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:28.251997 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:28.251997 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:28.252326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:28.252024 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:29.251785 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:29.251750 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:29.251785 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:29.251785 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:29.251785 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:29.252230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:29.251798 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:30.251504 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:30.251472 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:30.251504 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:30.251504 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:30.251504 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:30.251785 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:30.251532 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:31.251595 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:31.251559 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:31.251595 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:31.251595 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:31.251595 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:31.252068 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:31.251613 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:32.251350 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:32.251311 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:32.251350 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:32.251350 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:32.251350 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:32.251595 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:32.251364 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:33.251121 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:33.251090 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:33.251121 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:33.251121 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:33.251121 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:33.251549 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:33.251142 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:34.251410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:34.251370 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:34.251410 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:34.251410 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:34.251410 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:34.251976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:34.251439 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:34.306268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:34.306225 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:34.383109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:34.383074 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:35.251106 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:35.251071 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:35.251106 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:35.251106 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:35.251106 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:35.251349 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:35.251125 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:36.251817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:36.251782 2559 patch_prober.go:28] interesting pod/router-default-5d96df899f-2wt7h container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 08 11:15:36.251817 ip-10-0-140-148 kubenswrapper[2559]: [-]has-synced failed: reason withheld Jul 08 11:15:36.251817 ip-10-0-140-148 kubenswrapper[2559]: [+]process-running ok Jul 08 11:15:36.251817 ip-10-0-140-148 kubenswrapper[2559]: healthz check failed Jul 08 11:15:36.252258 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:36.251848 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5d96df899f-2wt7h" podUID="a02b54ca-d226-4494-bc5f-c3a28d197408" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 11:15:36.613037 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:36.612950 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:36.781811 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:36.781771 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:36.966300 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:36.966273 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:37.251755 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:37.251676 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:15:37.252851 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:37.252833 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d96df899f-2wt7h" Jul 08 11:15:37.782621 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:37.782588 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:37.919999 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:37.919962 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:40.048062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:40.048029 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:40.215548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:40.215513 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:42.723103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:42.723072 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:42.902319 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:42.902282 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:46.250447 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:46.250413 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:46.361220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:46.361180 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:49.649654 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:49.649610 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:49.754213 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:49.754166 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:49.852013 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:49.851979 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:50.706571 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:50.706533 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:51.518493 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:51.518461 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:15:51.616112 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:15:51.616079 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:07.131235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:07.131191 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:07.189214 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:07.189179 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:11.316512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:11.316479 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:11.410329 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:11.410296 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:25.356251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:25.356212 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:25.446251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:25.446223 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:25.507338 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:25.507305 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:26.266401 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:26.266364 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:26.324547 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:26.324519 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:56.212515 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:56.212477 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:57.986025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:57.985992 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:16:58.087936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:16:58.087901 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:17:54.920036 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:17:54.919997 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:17:54.985396 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:17:54.985356 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:18:04.039383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:18:04.039350 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:18:04.048949 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:18:04.048919 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:18:04.050288 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:18:04.050266 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 11:18:05.280281 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:18:05.280219 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:18:05.337261 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:18:05.337230 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:53.541042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:53.541003 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:54.778619 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:54.778585 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:55.483948 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:55.483896 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:55.586535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:55.586502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:55.638539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:55.638503 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:56.223934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:56.223902 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:19:58.726700 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:19:58.726665 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:20:53.593800 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:20:53.593708 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:20:53.695558 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:20:53.695523 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:08.643246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:08.643209 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:08.697837 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:08.697805 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:20.677006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:20.676962 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:20.746876 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:20.746842 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:26.270794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:26.270762 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:26.370931 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:26.370896 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:26.584164 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:26.584074 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:26.740684 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:26.740655 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:31.956420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:31.956380 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:32.047162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:32.047129 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:49.247962 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:49.247918 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:49.310577 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:49.310549 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:49.410240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:49.410199 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:49.675871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:49.675774 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:49.765698 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:49.765665 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:21:50.172990 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.172961 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd"] Jul 08 11:21:50.173234 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.173222 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" Jul 08 11:21:50.173283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.173235 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" Jul 08 11:21:50.173283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.173280 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e79a886-6964-48f8-b870-0a6ed3bd464a" containerName="registry" Jul 08 11:21:50.176235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.176221 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.178871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.178850 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 11:21:50.178871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.178861 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 11:21:50.179725 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.179712 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7psn8\"" Jul 08 11:21:50.185936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.185916 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd"] Jul 08 11:21:50.195358 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.195338 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.195458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.195367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.195458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.195409 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.296074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.296037 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.296542 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.296082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.296542 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.296148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.296708 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.296541 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.296708 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.296663 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.304336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.304309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.485764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.485730 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:21:50.617878 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.617856 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd"] Jul 08 11:21:50.620153 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:21:50.620123 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9e6057_cb02_4d45_9340_cf5c05268235.slice/crio-2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0 WatchSource:0}: Error finding container 2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0: Status 404 returned error can't find the container with id 2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0 Jul 08 11:21:50.622179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:50.622165 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 11:21:51.216906 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:51.216872 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerStarted","Data":"2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0"} Jul 08 11:21:59.947103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:21:59.947002 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:00.003858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:00.003785 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:01.258241 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:01.258211 2559 generic.go:356] "Generic (PLEG): container finished" podID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerID="b4aaf264651d5a98ea89ad97ca17c75904032fe35c6c19c9d481614da1d39349" exitCode=0 Jul 08 11:22:01.258614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:01.258302 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerDied","Data":"b4aaf264651d5a98ea89ad97ca17c75904032fe35c6c19c9d481614da1d39349"} Jul 08 11:22:03.267762 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:03.267734 2559 generic.go:356] "Generic (PLEG): container finished" podID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerID="1d9815268206be1af68e44b6d761edd854f6655a68cd368e8d4ec4ef2c36d077" exitCode=0 Jul 08 11:22:03.268089 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:03.267817 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerDied","Data":"1d9815268206be1af68e44b6d761edd854f6655a68cd368e8d4ec4ef2c36d077"} Jul 08 11:22:10.760828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:10.760786 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:10.817883 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:10.817846 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:11.302323 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:11.302291 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerStarted","Data":"5f920fb79719279399ba97fddc4fb7423ee205880bd71eed49f92e64b671ca77"} Jul 08 11:22:11.319798 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:11.319739 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" podStartSLOduration=1.887389972 podStartE2EDuration="21.31972311s" totalImagesPullingTime="19.432333138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:21:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:22:11.318327841 +0000 UTC m=+547.699974613" watchObservedRunningTime="2026-07-08 11:22:11.31972311 +0000 UTC m=+547.701369912" Jul 08 11:22:12.307970 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:12.307937 2559 generic.go:356] "Generic (PLEG): container finished" podID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerID="5f920fb79719279399ba97fddc4fb7423ee205880bd71eed49f92e64b671ca77" exitCode=0 Jul 08 11:22:12.308398 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:12.308033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerDied","Data":"5f920fb79719279399ba97fddc4fb7423ee205880bd71eed49f92e64b671ca77"} Jul 08 11:22:13.475601 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.475569 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:22:13.582900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.582804 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle\") pod \"0e9e6057-cb02-4d45-9340-cf5c05268235\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " Jul 08 11:22:13.582900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.582844 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util\") pod \"0e9e6057-cb02-4d45-9340-cf5c05268235\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " Jul 08 11:22:13.582900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.582868 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt\") pod \"0e9e6057-cb02-4d45-9340-cf5c05268235\" (UID: \"0e9e6057-cb02-4d45-9340-cf5c05268235\") " Jul 08 11:22:13.583504 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.583479 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle" (OuterVolumeSpecName: "bundle") pod "0e9e6057-cb02-4d45-9340-cf5c05268235" (UID: "0e9e6057-cb02-4d45-9340-cf5c05268235"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:13.585223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.585196 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt" (OuterVolumeSpecName: "kube-api-access-9b9kt") pod "0e9e6057-cb02-4d45-9340-cf5c05268235" (UID: "0e9e6057-cb02-4d45-9340-cf5c05268235"). InnerVolumeSpecName "kube-api-access-9b9kt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:22:13.586983 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.586962 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util" (OuterVolumeSpecName: "util") pod "0e9e6057-cb02-4d45-9340-cf5c05268235" (UID: "0e9e6057-cb02-4d45-9340-cf5c05268235"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:13.683917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.683886 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:13.683917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.683910 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e9e6057-cb02-4d45-9340-cf5c05268235-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:13.683917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:13.683919 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9kt\" (UniqueName: \"kubernetes.io/projected/0e9e6057-cb02-4d45-9340-cf5c05268235-kube-api-access-9b9kt\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:14.316858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:14.316827 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" Jul 08 11:22:14.316994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:14.316827 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409mvjd" event={"ID":"0e9e6057-cb02-4d45-9340-cf5c05268235","Type":"ContainerDied","Data":"2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0"} Jul 08 11:22:14.316994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:14.316963 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cc4ea41034025ff196e7f31231cd47cd6e71126a8ebe9a2c620e3d6cc7872d0" Jul 08 11:22:16.213038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:16.213007 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:17.008894 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:17.008859 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:18.234903 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.234864 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv"] Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235138 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="util" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235149 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="util" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235156 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="pull" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235161 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="pull" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235174 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="extract" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235180 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="extract" Jul 08 11:22:18.235299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.235246 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9e6057-cb02-4d45-9340-cf5c05268235" containerName="extract" Jul 08 11:22:18.237714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.237699 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.241728 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.241708 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 11:22:18.241837 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.241810 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tm9xh\"" Jul 08 11:22:18.241886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.241832 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:22:18.256669 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.256628 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv"] Jul 08 11:22:18.317433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.317400 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bf4d08-d62e-474d-ade0-1d22779ef92a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.317433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.317441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xzfv\" (UniqueName: \"kubernetes.io/projected/47bf4d08-d62e-474d-ade0-1d22779ef92a-kube-api-access-5xzfv\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.418788 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.418749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bf4d08-d62e-474d-ade0-1d22779ef92a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.418964 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.418799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xzfv\" (UniqueName: \"kubernetes.io/projected/47bf4d08-d62e-474d-ade0-1d22779ef92a-kube-api-access-5xzfv\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.419092 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.419072 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47bf4d08-d62e-474d-ade0-1d22779ef92a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.427572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.427550 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xzfv\" (UniqueName: \"kubernetes.io/projected/47bf4d08-d62e-474d-ade0-1d22779ef92a-kube-api-access-5xzfv\") pod \"cert-manager-operator-controller-manager-55c444bf86-bq2rv\" (UID: \"47bf4d08-d62e-474d-ade0-1d22779ef92a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.547016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.546916 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" Jul 08 11:22:18.687377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:18.687348 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv"] Jul 08 11:22:18.689141 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:18.689101 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47bf4d08_d62e_474d_ade0_1d22779ef92a.slice/crio-a2b9d055b66e2a1b4b01df370e2e1fadc3c5de1bb1035a40757a7fabf8037926 WatchSource:0}: Error finding container a2b9d055b66e2a1b4b01df370e2e1fadc3c5de1bb1035a40757a7fabf8037926: Status 404 returned error can't find the container with id a2b9d055b66e2a1b4b01df370e2e1fadc3c5de1bb1035a40757a7fabf8037926 Jul 08 11:22:19.338100 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:19.338061 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" event={"ID":"47bf4d08-d62e-474d-ade0-1d22779ef92a","Type":"ContainerStarted","Data":"a2b9d055b66e2a1b4b01df370e2e1fadc3c5de1bb1035a40757a7fabf8037926"} Jul 08 11:22:19.813550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:19.813508 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:19.922747 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:19.922712 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:20.344282 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:20.344238 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" event={"ID":"47bf4d08-d62e-474d-ade0-1d22779ef92a","Type":"ContainerStarted","Data":"4bfe9f156a9c859d5264780f301f84f7cb37c661dc8894f55e251b928bd77c31"} Jul 08 11:22:20.368689 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:20.368609 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bq2rv" podStartSLOduration=0.804498575 podStartE2EDuration="2.368591653s" totalImagesPullingTime="1.564093078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:22:20.366994808 +0000 UTC m=+556.748641580" watchObservedRunningTime="2026-07-08 11:22:20.368591653 +0000 UTC m=+556.750238425" Jul 08 11:22:20.877597 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:20.877542 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:21.025787 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.025754 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:21.635051 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.635019 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx"] Jul 08 11:22:21.638561 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.638542 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.641324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.641297 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 11:22:21.641324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.641297 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7psn8\"" Jul 08 11:22:21.642052 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.642038 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 11:22:21.648349 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.648325 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx"] Jul 08 11:22:21.748763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.748723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.748763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.748769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.748977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.748878 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79md4\" (UniqueName: \"kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.850047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.850013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.850047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.850050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.850290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.850107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79md4\" (UniqueName: \"kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.850405 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.850382 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.850488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.850470 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.860284 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.860252 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79md4\" (UniqueName: \"kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.948114 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.948042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:21.987104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:21.987064 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:22.080788 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.080764 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx"] Jul 08 11:22:22.082653 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:22.082605 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b58c170_5711_4bbb_9c09_96ca00507e60.slice/crio-8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21 WatchSource:0}: Error finding container 8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21: Status 404 returned error can't find the container with id 8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21 Jul 08 11:22:22.087661 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.087601 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:22.120383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.120356 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8n94g"] Jul 08 11:22:22.124748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.124727 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.127257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.127231 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 11:22:22.127367 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.127336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-d5qw4\"" Jul 08 11:22:22.127438 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.127383 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 11:22:22.133581 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.133520 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8n94g"] Jul 08 11:22:22.254954 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.254921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzc6k\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-kube-api-access-rzc6k\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.255109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.254980 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.355512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.355479 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzc6k\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-kube-api-access-rzc6k\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.355713 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.355531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.356482 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.356456 2559 generic.go:356] "Generic (PLEG): container finished" podID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerID="38032af2624510f4190641cff8410a5460eda4e7547c7fa4c7ed599eed02a1ce" exitCode=0 Jul 08 11:22:22.356597 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.356490 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" event={"ID":"9b58c170-5711-4bbb-9c09-96ca00507e60","Type":"ContainerDied","Data":"38032af2624510f4190641cff8410a5460eda4e7547c7fa4c7ed599eed02a1ce"} Jul 08 11:22:22.356597 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.356527 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" event={"ID":"9b58c170-5711-4bbb-9c09-96ca00507e60","Type":"ContainerStarted","Data":"8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21"} Jul 08 11:22:22.363717 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.363694 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.364111 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.364081 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzc6k\" (UniqueName: \"kubernetes.io/projected/ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d-kube-api-access-rzc6k\") pod \"cert-manager-webhook-6486ffd997-8n94g\" (UID: \"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.439310 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.439277 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:22.567540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:22.567513 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8n94g"] Jul 08 11:22:22.569508 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:22.569485 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee4f5e30_2e9f_4973_ae6b_76d9e9ddd77d.slice/crio-f9c9c6580a29eadd43b12896f9425b090e01f878a30ac98942ce9ac4932784ea WatchSource:0}: Error finding container f9c9c6580a29eadd43b12896f9425b090e01f878a30ac98942ce9ac4932784ea: Status 404 returned error can't find the container with id f9c9c6580a29eadd43b12896f9425b090e01f878a30ac98942ce9ac4932784ea Jul 08 11:22:23.363140 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:23.363103 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" event={"ID":"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d","Type":"ContainerStarted","Data":"f9c9c6580a29eadd43b12896f9425b090e01f878a30ac98942ce9ac4932784ea"} Jul 08 11:22:24.199535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:24.199501 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:24.369614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:24.369586 2559 generic.go:356] "Generic (PLEG): container finished" podID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerID="8a97711a0650bcd9cec67b7e9741c9907cb0e374bd890695ec8d598eb06f0a91" exitCode=0 Jul 08 11:22:24.369956 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:24.369670 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" event={"ID":"9b58c170-5711-4bbb-9c09-96ca00507e60","Type":"ContainerDied","Data":"8a97711a0650bcd9cec67b7e9741c9907cb0e374bd890695ec8d598eb06f0a91"} Jul 08 11:22:24.396307 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:24.396258 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:25.375700 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:25.375663 2559 generic.go:356] "Generic (PLEG): container finished" podID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerID="57f6e45618208689f8330f0796eea39224275180a00fb566790e77bf68268eb6" exitCode=0 Jul 08 11:22:25.376063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:25.375743 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" event={"ID":"9b58c170-5711-4bbb-9c09-96ca00507e60","Type":"ContainerDied","Data":"57f6e45618208689f8330f0796eea39224275180a00fb566790e77bf68268eb6"} Jul 08 11:22:26.504881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.504858 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:26.594938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.594910 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util\") pod \"9b58c170-5711-4bbb-9c09-96ca00507e60\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " Jul 08 11:22:26.595108 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.594961 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle\") pod \"9b58c170-5711-4bbb-9c09-96ca00507e60\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " Jul 08 11:22:26.595108 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.595009 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79md4\" (UniqueName: \"kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4\") pod \"9b58c170-5711-4bbb-9c09-96ca00507e60\" (UID: \"9b58c170-5711-4bbb-9c09-96ca00507e60\") " Jul 08 11:22:26.595383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.595357 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle" (OuterVolumeSpecName: "bundle") pod "9b58c170-5711-4bbb-9c09-96ca00507e60" (UID: "9b58c170-5711-4bbb-9c09-96ca00507e60"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:26.597355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.597334 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4" (OuterVolumeSpecName: "kube-api-access-79md4") pod "9b58c170-5711-4bbb-9c09-96ca00507e60" (UID: "9b58c170-5711-4bbb-9c09-96ca00507e60"). InnerVolumeSpecName "kube-api-access-79md4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:22:26.600115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.600092 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util" (OuterVolumeSpecName: "util") pod "9b58c170-5711-4bbb-9c09-96ca00507e60" (UID: "9b58c170-5711-4bbb-9c09-96ca00507e60"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:26.614137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.614107 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:26.696291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.696203 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79md4\" (UniqueName: \"kubernetes.io/projected/9b58c170-5711-4bbb-9c09-96ca00507e60-kube-api-access-79md4\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:26.696291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.696230 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:26.696291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.696239 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b58c170-5711-4bbb-9c09-96ca00507e60-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:26.714045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:26.714012 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:27.386039 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:27.386006 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" event={"ID":"9b58c170-5711-4bbb-9c09-96ca00507e60","Type":"ContainerDied","Data":"8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21"} Jul 08 11:22:27.386039 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:27.386038 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkwsxx" Jul 08 11:22:27.386213 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:27.386051 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ff89d2430db8f5725ca71f4892e2959a5d6307ee1fcfe4e97bda5824685dc21" Jul 08 11:22:30.025696 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:30.025657 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:30.155498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:30.155467 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:32.391526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:32.391493 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:34.789240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789208 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5sdv"] Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789609 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="extract" Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789626 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="extract" Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789660 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="pull" Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789668 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="pull" Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789690 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="util" Jul 08 11:22:34.789714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789699 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="util" Jul 08 11:22:34.790000 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.789777 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b58c170-5711-4bbb-9c09-96ca00507e60" containerName="extract" Jul 08 11:22:34.792740 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.792720 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:34.795063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.795048 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zq58m\"" Jul 08 11:22:34.801814 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.801790 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5sdv"] Jul 08 11:22:34.960503 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.960467 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:34.960705 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:34.960548 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc94x\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-kube-api-access-dc94x\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.061622 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.061534 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc94x\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-kube-api-access-dc94x\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.061622 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.061573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.069988 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.069964 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-bound-sa-token\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.070173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.070153 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc94x\" (UniqueName: \"kubernetes.io/projected/3264b708-9d2c-4006-9d8d-35020060e21a-kube-api-access-dc94x\") pod \"cert-manager-55b4d954d5-j5sdv\" (UID: \"3264b708-9d2c-4006-9d8d-35020060e21a\") " pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.101944 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.101916 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j5sdv" Jul 08 11:22:35.232186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.232160 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j5sdv"] Jul 08 11:22:35.234602 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:35.234576 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3264b708_9d2c_4006_9d8d_35020060e21a.slice/crio-77f7c6ab718ab2955f5ff38ccc5bb530eef8b3523eb672897a6adba454499b15 WatchSource:0}: Error finding container 77f7c6ab718ab2955f5ff38ccc5bb530eef8b3523eb672897a6adba454499b15: Status 404 returned error can't find the container with id 77f7c6ab718ab2955f5ff38ccc5bb530eef8b3523eb672897a6adba454499b15 Jul 08 11:22:35.419181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.419099 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j5sdv" event={"ID":"3264b708-9d2c-4006-9d8d-35020060e21a","Type":"ContainerStarted","Data":"77f7c6ab718ab2955f5ff38ccc5bb530eef8b3523eb672897a6adba454499b15"} Jul 08 11:22:35.653827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.653785 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:35.758859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:35.758824 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:37.298871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.298836 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv"] Jul 08 11:22:37.302570 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.302551 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.305203 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.305180 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 11:22:37.305390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.305367 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 11:22:37.305390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.305377 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7psn8\"" Jul 08 11:22:37.311282 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.310546 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv"] Jul 08 11:22:37.479254 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.479219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b4gz\" (UniqueName: \"kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.479432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.479372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.479491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.479461 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.580626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.580553 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.580626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.580599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.580823 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.580630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b4gz\" (UniqueName: \"kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.581020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.580993 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.581061 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.581000 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.591451 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.591425 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b4gz\" (UniqueName: \"kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.614346 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.614321 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:37.806482 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.805908 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv"] Jul 08 11:22:37.810178 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:37.810151 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c20e705_7759_4178_b7f2_be4713da80c7.slice/crio-38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a WatchSource:0}: Error finding container 38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a: Status 404 returned error can't find the container with id 38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a Jul 08 11:22:37.984508 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:37.984469 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:38.435435 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.435392 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j5sdv" event={"ID":"3264b708-9d2c-4006-9d8d-35020060e21a","Type":"ContainerStarted","Data":"839f6ec8dc2b6b50ac340657f93047f86bca61d79a5fcd352237b05ebd6c4a90"} Jul 08 11:22:38.436971 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.436950 2559 generic.go:356] "Generic (PLEG): container finished" podID="3c20e705-7759-4178-b7f2-be4713da80c7" containerID="5f573726afd5e7e17d1665d9dec57717e217f911631b366d86f56f17a91da2ab" exitCode=0 Jul 08 11:22:38.437058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.437033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" event={"ID":"3c20e705-7759-4178-b7f2-be4713da80c7","Type":"ContainerDied","Data":"5f573726afd5e7e17d1665d9dec57717e217f911631b366d86f56f17a91da2ab"} Jul 08 11:22:38.437130 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.437087 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" event={"ID":"3c20e705-7759-4178-b7f2-be4713da80c7","Type":"ContainerStarted","Data":"38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a"} Jul 08 11:22:38.438518 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.438500 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" event={"ID":"ee4f5e30-2e9f-4973-ae6b-76d9e9ddd77d","Type":"ContainerStarted","Data":"a95698499e3c488ed91e36c29528857af5019e87ecf5a3722d3adfef60622549"} Jul 08 11:22:38.438661 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.438648 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:38.454198 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.454152 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j5sdv" podStartSLOduration=2.026831722 podStartE2EDuration="4.454137668s" totalImagesPullingTime="2.427305946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:22:38.452878208 +0000 UTC m=+574.834524986" watchObservedRunningTime="2026-07-08 11:22:38.454137668 +0000 UTC m=+574.835784438" Jul 08 11:22:38.495756 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:38.495704 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" podStartSLOduration=1.402353244 podStartE2EDuration="16.495687701s" totalImagesPullingTime="15.093334457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:22:38.494255282 +0000 UTC m=+574.875902055" watchObservedRunningTime="2026-07-08 11:22:38.495687701 +0000 UTC m=+574.877334471" Jul 08 11:22:39.444825 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:39.444745 2559 generic.go:356] "Generic (PLEG): container finished" podID="3c20e705-7759-4178-b7f2-be4713da80c7" containerID="2ceb45bfe105a8b8bf7aaf2bffc34b1a86ed595e5977d7aac37c38b19a8d11a4" exitCode=0 Jul 08 11:22:39.445215 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:39.444834 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" event={"ID":"3c20e705-7759-4178-b7f2-be4713da80c7","Type":"ContainerDied","Data":"2ceb45bfe105a8b8bf7aaf2bffc34b1a86ed595e5977d7aac37c38b19a8d11a4"} Jul 08 11:22:40.451149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:40.451112 2559 generic.go:356] "Generic (PLEG): container finished" podID="3c20e705-7759-4178-b7f2-be4713da80c7" containerID="e74ba882dcb803c23d3811945712e71c5a8bab2c5e03ec96b2d3d1e7566324d9" exitCode=0 Jul 08 11:22:40.451509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:40.451200 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" event={"ID":"3c20e705-7759-4178-b7f2-be4713da80c7","Type":"ContainerDied","Data":"e74ba882dcb803c23d3811945712e71c5a8bab2c5e03ec96b2d3d1e7566324d9"} Jul 08 11:22:41.592930 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.592903 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:41.716330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.716232 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle\") pod \"3c20e705-7759-4178-b7f2-be4713da80c7\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " Jul 08 11:22:41.716330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.716293 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b4gz\" (UniqueName: \"kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz\") pod \"3c20e705-7759-4178-b7f2-be4713da80c7\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " Jul 08 11:22:41.716330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.716331 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util\") pod \"3c20e705-7759-4178-b7f2-be4713da80c7\" (UID: \"3c20e705-7759-4178-b7f2-be4713da80c7\") " Jul 08 11:22:41.717183 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.717156 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle" (OuterVolumeSpecName: "bundle") pod "3c20e705-7759-4178-b7f2-be4713da80c7" (UID: "3c20e705-7759-4178-b7f2-be4713da80c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:41.718591 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.718571 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz" (OuterVolumeSpecName: "kube-api-access-5b4gz") pod "3c20e705-7759-4178-b7f2-be4713da80c7" (UID: "3c20e705-7759-4178-b7f2-be4713da80c7"). InnerVolumeSpecName "kube-api-access-5b4gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:22:41.721955 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.721935 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util" (OuterVolumeSpecName: "util") pod "3c20e705-7759-4178-b7f2-be4713da80c7" (UID: "3c20e705-7759-4178-b7f2-be4713da80c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:22:41.817556 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.817520 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:41.817556 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.817552 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c20e705-7759-4178-b7f2-be4713da80c7-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:41.817556 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:41.817563 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b4gz\" (UniqueName: \"kubernetes.io/projected/3c20e705-7759-4178-b7f2-be4713da80c7-kube-api-access-5b4gz\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:22:42.462617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:42.462575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" event={"ID":"3c20e705-7759-4178-b7f2-be4713da80c7","Type":"ContainerDied","Data":"38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a"} Jul 08 11:22:42.462617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:42.462601 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgdcjv" Jul 08 11:22:42.462903 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:42.462681 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38bc976be9916991e281964c6c4b3c57f0d698550f0be7e71c12b4c7f3ef370a" Jul 08 11:22:43.009583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:43.009553 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:43.070726 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:43.070689 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:44.447853 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:44.447816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8n94g" Jul 08 11:22:51.709703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:51.709666 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:51.792817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:51.792784 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:52.443904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.443870 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:52.689033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.688999 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:52.793459 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.793427 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:52.818822 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.818787 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf"] Jul 08 11:22:52.819074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819057 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="util" Jul 08 11:22:52.819074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819073 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="util" Jul 08 11:22:52.819223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819095 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="pull" Jul 08 11:22:52.819223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819100 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="pull" Jul 08 11:22:52.819223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819108 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="extract" Jul 08 11:22:52.819223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819113 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="extract" Jul 08 11:22:52.819223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.819157 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c20e705-7759-4178-b7f2-be4713da80c7" containerName="extract" Jul 08 11:22:52.821393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.821376 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.828321 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.827964 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 11:22:52.828418 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.828322 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bhz7d\"" Jul 08 11:22:52.828418 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.828335 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 11:22:52.828613 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.828596 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 11:22:52.828969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.828952 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 11:22:52.856766 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.856737 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf"] Jul 08 11:22:52.901142 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.901107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.901328 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.901148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trgj8\" (UniqueName: \"kubernetes.io/projected/5fe216ce-fb37-48af-84b5-5124b2322006-kube-api-access-trgj8\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.901328 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.901253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-manifests\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.901328 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.901286 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-tmp\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.901328 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.901309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:52.979483 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:52.979447 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.001624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.001596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-manifests\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.001624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.001651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-tmp\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.001842 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.001675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.001842 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.001707 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.001842 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.001734 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trgj8\" (UniqueName: \"kubernetes.io/projected/5fe216ce-fb37-48af-84b5-5124b2322006-kube-api-access-trgj8\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.002047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.002025 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-manifests\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.004247 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.004216 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5fe216ce-fb37-48af-84b5-5124b2322006-tmp\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.004355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.004257 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.004429 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.004415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5fe216ce-fb37-48af-84b5-5124b2322006-webhook-cert\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.010701 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.010679 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trgj8\" (UniqueName: \"kubernetes.io/projected/5fe216ce-fb37-48af-84b5-5124b2322006-kube-api-access-trgj8\") pod \"opendatahub-operator-controller-manager-79c9b86684-ltxbf\" (UID: \"5fe216ce-fb37-48af-84b5-5124b2322006\") " pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.090934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.090839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.132091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.132045 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:53.179390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.179346 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.271587 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.271561 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf"] Jul 08 11:22:53.272971 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:22:53.272940 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe216ce_fb37_48af_84b5_5124b2322006.slice/crio-36238ce5f3aac3e2af3a08bd5383663e4625848dd467e842725c4d571dfd4f9a WatchSource:0}: Error finding container 36238ce5f3aac3e2af3a08bd5383663e4625848dd467e842725c4d571dfd4f9a: Status 404 returned error can't find the container with id 36238ce5f3aac3e2af3a08bd5383663e4625848dd467e842725c4d571dfd4f9a Jul 08 11:22:53.273507 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.273479 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.330242 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.330206 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.430687 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.430571 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:22:53.513854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:53.513821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" event={"ID":"5fe216ce-fb37-48af-84b5-5124b2322006","Type":"ContainerStarted","Data":"36238ce5f3aac3e2af3a08bd5383663e4625848dd467e842725c4d571dfd4f9a"} Jul 08 11:22:57.537497 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:57.537449 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" event={"ID":"5fe216ce-fb37-48af-84b5-5124b2322006","Type":"ContainerStarted","Data":"92b83c8685dd753d964da8a9703327b78ef2177ea401167d55c893bf4e9e46f3"} Jul 08 11:22:58.543397 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:58.543364 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fe216ce-fb37-48af-84b5-5124b2322006" containerID="92b83c8685dd753d964da8a9703327b78ef2177ea401167d55c893bf4e9e46f3" exitCode=0 Jul 08 11:22:58.543397 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:58.543401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" event={"ID":"5fe216ce-fb37-48af-84b5-5124b2322006","Type":"ContainerDied","Data":"92b83c8685dd753d964da8a9703327b78ef2177ea401167d55c893bf4e9e46f3"} Jul 08 11:22:59.549345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:59.549310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" event={"ID":"5fe216ce-fb37-48af-84b5-5124b2322006","Type":"ContainerStarted","Data":"5d9b11fa209ed20771208d14480516e50435ab1b5c7510fb3b266ecc1ac8617f"} Jul 08 11:22:59.549772 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:59.549383 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:22:59.572153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:22:59.572098 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" podStartSLOduration=2.649535116 podStartE2EDuration="7.572081025s" totalImagesPullingTime="3.846633608s" totalInitContainerRuntime="1.075912301s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:22:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:22:59.570876831 +0000 UTC m=+595.952523626" watchObservedRunningTime="2026-07-08 11:22:59.572081025 +0000 UTC m=+595.953727796" Jul 08 11:23:00.206182 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:00.206150 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:00.308268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:00.308226 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:02.849075 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:02.849022 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:04.070912 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:04.070883 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:23:04.075832 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:04.075811 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:23:04.357929 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:04.357814 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:04.454535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:04.454505 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:10.557175 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.557142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79c9b86684-ltxbf" Jul 08 11:23:10.784014 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.783979 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5"] Jul 08 11:23:10.787113 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.787094 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.801056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.801029 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 11:23:10.802208 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.802187 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 11:23:10.802313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.802220 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 11:23:10.802313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.802187 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 11:23:10.802313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.802222 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 11:23:10.802455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.802240 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xmk2w\"" Jul 08 11:23:10.808313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.808247 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5"] Jul 08 11:23:10.822936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.822903 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsnb\" (UniqueName: \"kubernetes.io/projected/aa0b7cec-8c6a-4284-90b0-320d273fe156-kube-api-access-7lsnb\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.822936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.822939 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa0b7cec-8c6a-4284-90b0-320d273fe156-manager-config\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.823138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.822965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.823138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.822984 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-metrics-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.923371 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.923333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsnb\" (UniqueName: \"kubernetes.io/projected/aa0b7cec-8c6a-4284-90b0-320d273fe156-kube-api-access-7lsnb\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.923538 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.923378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa0b7cec-8c6a-4284-90b0-320d273fe156-manager-config\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.923538 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.923424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.923538 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.923449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-metrics-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.924283 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.924204 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa0b7cec-8c6a-4284-90b0-320d273fe156-manager-config\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.926009 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.925979 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.926251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.926111 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b7cec-8c6a-4284-90b0-320d273fe156-metrics-cert\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:10.931732 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:10.931702 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsnb\" (UniqueName: \"kubernetes.io/projected/aa0b7cec-8c6a-4284-90b0-320d273fe156-kube-api-access-7lsnb\") pod \"lws-controller-manager-574bb547d7-h4ch5\" (UID: \"aa0b7cec-8c6a-4284-90b0-320d273fe156\") " pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:11.097757 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:11.097663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:11.265841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:11.265811 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5"] Jul 08 11:23:11.267845 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:23:11.267817 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0b7cec_8c6a_4284_90b0_320d273fe156.slice/crio-cba225928c90e64c05ffb17875fe7625e7f5231f8a1df0076034ef42fe91f5de WatchSource:0}: Error finding container cba225928c90e64c05ffb17875fe7625e7f5231f8a1df0076034ef42fe91f5de: Status 404 returned error can't find the container with id cba225928c90e64c05ffb17875fe7625e7f5231f8a1df0076034ef42fe91f5de Jul 08 11:23:11.602961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:11.602924 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" event={"ID":"aa0b7cec-8c6a-4284-90b0-320d273fe156","Type":"ContainerStarted","Data":"cba225928c90e64c05ffb17875fe7625e7f5231f8a1df0076034ef42fe91f5de"} Jul 08 11:23:12.453095 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:12.453056 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:12.571942 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:12.571901 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.015250 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.015220 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.192510 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.192427 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.336471 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.336440 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.437742 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.437688 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.532093 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.532063 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:13.615238 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.615191 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" event={"ID":"aa0b7cec-8c6a-4284-90b0-320d273fe156","Type":"ContainerStarted","Data":"3e11f8ef0c2286c6053d797079041707628f59154a7d2b16e93e9928bf9a8232"} Jul 08 11:23:13.615418 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.615269 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:13.632497 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:13.632444 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" podStartSLOduration=2.073217544 podStartE2EDuration="3.632427938s" totalImagesPullingTime="1.559210394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:23:13.63102476 +0000 UTC m=+610.012671532" watchObservedRunningTime="2026-07-08 11:23:13.632427938 +0000 UTC m=+610.014074709" Jul 08 11:23:14.192562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:14.192526 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:14.284898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:14.284821 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:14.468001 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:14.467965 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:14.533137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:14.533097 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:15.545018 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:15.544976 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:20.563462 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:20.563425 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:21.238058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.238025 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4"] Jul 08 11:23:21.241573 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.241550 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.244789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.244769 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5m568\"" Jul 08 11:23:21.244895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.244817 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 11:23:21.276758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.276730 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4"] Jul 08 11:23:21.284040 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.284012 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:21.307679 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307654 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.307804 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.307804 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.307902 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.307902 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.307902 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307840 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.308016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.308016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307939 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh2v5\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-kube-api-access-nh2v5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.308016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.307958 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.381498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.381461 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:21.408620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408702 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh2v5\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-kube-api-access-nh2v5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.408817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408796 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408837 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.408889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.409116 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409289 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.409200 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409289 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.409271 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.409338 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.409607 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.409587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.411525 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.411502 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.411653 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.411583 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.417548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.417524 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.418315 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.418289 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh2v5\" (UniqueName: \"kubernetes.io/projected/7067d7ad-0ca3-4806-a452-f528d6ac09d9-kube-api-access-nh2v5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4dxcq4\" (UID: \"7067d7ad-0ca3-4806-a452-f528d6ac09d9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.476789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.476758 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:21.551136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.551053 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:21.705614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:21.705591 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4"] Jul 08 11:23:21.707036 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:23:21.707014 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7067d7ad_0ca3_4806_a452_f528d6ac09d9.slice/crio-63fe9a25bfe78f421ac7f151c9631e7d5a8d771a8c40972b358661c0e4ea21ca WatchSource:0}: Error finding container 63fe9a25bfe78f421ac7f151c9631e7d5a8d771a8c40972b358661c0e4ea21ca: Status 404 returned error can't find the container with id 63fe9a25bfe78f421ac7f151c9631e7d5a8d771a8c40972b358661c0e4ea21ca Jul 08 11:23:22.668622 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:22.668579 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" event={"ID":"7067d7ad-0ca3-4806-a452-f528d6ac09d9","Type":"ContainerStarted","Data":"63fe9a25bfe78f421ac7f151c9631e7d5a8d771a8c40972b358661c0e4ea21ca"} Jul 08 11:23:24.305277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.305240 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:23:24.305520 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.305315 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:23:24.305520 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.305343 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:23:24.517198 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.517135 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:24.609707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.609520 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lngzl"] Jul 08 11:23:24.613452 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.613432 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.617481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.617453 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 11:23:24.617569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.617497 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-khwv9\"" Jul 08 11:23:24.622405 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.622379 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-574bb547d7-h4ch5" Jul 08 11:23:24.626229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.626203 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n"] Jul 08 11:23:24.629364 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.629348 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.632854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.632588 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 11:23:24.632854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.632616 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5mtrl\"" Jul 08 11:23:24.635809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.635756 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lngzl"] Jul 08 11:23:24.644059 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.644038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26rw5\" (UniqueName: \"kubernetes.io/projected/f996741e-93d9-49a1-8003-79ca3775ba11-kube-api-access-26rw5\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.644222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.644115 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.658655 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.658609 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:24.658857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.658835 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n"] Jul 08 11:23:24.682521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.682487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" event={"ID":"7067d7ad-0ca3-4806-a452-f528d6ac09d9","Type":"ContainerStarted","Data":"c8c36f983a93e992405751e19a2f70e71ba9081d6d0d1f7f0491379c2f4283e6"} Jul 08 11:23:24.723229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.723175 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" podStartSLOduration=1.127345367 podStartE2EDuration="3.723159601s" totalImagesPullingTime="2.595814234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:23:24.721126267 +0000 UTC m=+621.102773040" watchObservedRunningTime="2026-07-08 11:23:24.723159601 +0000 UTC m=+621.104806371" Jul 08 11:23:24.745137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.745107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26rw5\" (UniqueName: \"kubernetes.io/projected/f996741e-93d9-49a1-8003-79ca3775ba11-kube-api-access-26rw5\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.745295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.745201 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.745295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.745247 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.745295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.745288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ldfk\" (UniqueName: \"kubernetes.io/projected/34a7f79c-7d3d-4e08-8db2-78e705308a85-kube-api-access-7ldfk\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.746004 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:23:24.745983 2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 11:23:24.746060 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:23:24.746052 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert podName:f996741e-93d9-49a1-8003-79ca3775ba11 nodeName:}" failed. No retries permitted until 2026-07-08 11:23:25.246033582 +0000 UTC m=+621.627680347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert") pod "kserve-controller-manager-d6f4b65b8-lngzl" (UID: "f996741e-93d9-49a1-8003-79ca3775ba11") : secret "kserve-webhook-server-cert" not found Jul 08 11:23:24.759952 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.759926 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26rw5\" (UniqueName: \"kubernetes.io/projected/f996741e-93d9-49a1-8003-79ca3775ba11-kube-api-access-26rw5\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:24.845911 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.845809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ldfk\" (UniqueName: \"kubernetes.io/projected/34a7f79c-7d3d-4e08-8db2-78e705308a85-kube-api-access-7ldfk\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.846104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.845949 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.846104 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:23:24.846062 2559 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 08 11:23:24.846234 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:23:24.846139 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert podName:34a7f79c-7d3d-4e08-8db2-78e705308a85 nodeName:}" failed. No retries permitted until 2026-07-08 11:23:25.346117788 +0000 UTC m=+621.727764548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert") pod "llmisvc-controller-manager-777c6bd7d5-zfg2n" (UID: "34a7f79c-7d3d-4e08-8db2-78e705308a85") : secret "llmisvc-webhook-server-cert" not found Jul 08 11:23:24.858907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.858873 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ldfk\" (UniqueName: \"kubernetes.io/projected/34a7f79c-7d3d-4e08-8db2-78e705308a85-kube-api-access-7ldfk\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:24.859989 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:24.859921 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:25.181614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.181524 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:25.248461 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.248432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:25.250933 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.250910 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f996741e-93d9-49a1-8003-79ca3775ba11-cert\") pod \"kserve-controller-manager-d6f4b65b8-lngzl\" (UID: \"f996741e-93d9-49a1-8003-79ca3775ba11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:25.349614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.349575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:25.352109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.352086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34a7f79c-7d3d-4e08-8db2-78e705308a85-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zfg2n\" (UID: \"34a7f79c-7d3d-4e08-8db2-78e705308a85\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:25.527823 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.527788 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:25.541103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.541072 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:25.551434 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.551404 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:25.557354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.557328 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:25.672365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.672338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lngzl"] Jul 08 11:23:25.675258 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:23:25.675223 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf996741e_93d9_49a1_8003_79ca3775ba11.slice/crio-a3c5312313ef20dec30fa08ea0cacb6061787d14a01c10a5cad1566ee24e9417 WatchSource:0}: Error finding container a3c5312313ef20dec30fa08ea0cacb6061787d14a01c10a5cad1566ee24e9417: Status 404 returned error can't find the container with id a3c5312313ef20dec30fa08ea0cacb6061787d14a01c10a5cad1566ee24e9417 Jul 08 11:23:25.688232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.688204 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" event={"ID":"f996741e-93d9-49a1-8003-79ca3775ba11","Type":"ContainerStarted","Data":"a3c5312313ef20dec30fa08ea0cacb6061787d14a01c10a5cad1566ee24e9417"} Jul 08 11:23:25.688535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.688514 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:25.689557 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.689539 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4dxcq4" Jul 08 11:23:25.718494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.718467 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n"] Jul 08 11:23:25.719821 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:23:25.719800 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34a7f79c_7d3d_4e08_8db2_78e705308a85.slice/crio-5edc316c7a8df50e60181273c528f3c5623c28c38dbf4a857054fb47001544cc WatchSource:0}: Error finding container 5edc316c7a8df50e60181273c528f3c5623c28c38dbf4a857054fb47001544cc: Status 404 returned error can't find the container with id 5edc316c7a8df50e60181273c528f3c5623c28c38dbf4a857054fb47001544cc Jul 08 11:23:25.871627 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.871537 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:25.961521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:25.961482 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:26.030935 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:26.030894 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:26.695073 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:26.695042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" event={"ID":"34a7f79c-7d3d-4e08-8db2-78e705308a85","Type":"ContainerStarted","Data":"5edc316c7a8df50e60181273c528f3c5623c28c38dbf4a857054fb47001544cc"} Jul 08 11:23:28.709318 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:28.709284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" event={"ID":"f996741e-93d9-49a1-8003-79ca3775ba11","Type":"ContainerStarted","Data":"b3bbfd9ace0aa4e152b2ba4836228fa22c65c0f7e5a18ad00e9b010fa5f1c3ea"} Jul 08 11:23:28.709733 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:28.709404 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:23:28.728743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:28.728689 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" podStartSLOduration=1.848803108 podStartE2EDuration="4.728667771s" totalImagesPullingTime="2.879864663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:23:28.728157945 +0000 UTC m=+625.109804716" watchObservedRunningTime="2026-07-08 11:23:28.728667771 +0000 UTC m=+625.110314543" Jul 08 11:23:29.714977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:29.714941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" event={"ID":"34a7f79c-7d3d-4e08-8db2-78e705308a85","Type":"ContainerStarted","Data":"a1edfb129a0a3acff31373a13b7c422728ac8bf69ee458d7add074f996a98b4f"} Jul 08 11:23:29.715374 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:29.715183 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:23:29.731833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:29.731780 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" podStartSLOduration=2.243253931 podStartE2EDuration="5.731762863s" totalImagesPullingTime="3.488508932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:23:29.730886848 +0000 UTC m=+626.112533620" watchObservedRunningTime="2026-07-08 11:23:29.731762863 +0000 UTC m=+626.113409634" Jul 08 11:23:33.228809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:33.228764 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:33.329991 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:33.329942 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:23:59.720766 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:23:59.720732 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lngzl" Jul 08 11:24:00.722194 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:00.722166 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zfg2n" Jul 08 11:24:14.008595 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:14.008550 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:14.084475 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:14.084445 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:14.163255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:14.163215 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:26.339125 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:26.339088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:26.398764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:26.398710 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:29.405531 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.405494 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp"] Jul 08 11:24:29.409011 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.408994 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.411314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.411281 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 11:24:29.411314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.411287 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qq5gw\"" Jul 08 11:24:29.412199 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.412184 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 11:24:29.416102 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.416081 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp"] Jul 08 11:24:29.481196 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.481163 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs54c\" (UniqueName: \"kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.481196 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.481200 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.481402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.481280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.582289 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.582250 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.582477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.582337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs54c\" (UniqueName: \"kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.582477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.582369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.582710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.582688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.582788 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.582717 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.590664 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.590610 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs54c\" (UniqueName: \"kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.719080 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.719047 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:29.851071 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.851044 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp"] Jul 08 11:24:29.853080 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:29.853050 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc9a805_a906_471c_9573_9355c0fd9263.slice/crio-0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e WatchSource:0}: Error finding container 0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e: Status 404 returned error can't find the container with id 0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e Jul 08 11:24:29.991335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.991240 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fc9a805-a906-471c-9573-9355c0fd9263" containerID="2f29e475e84ade6ebedca001278e0c0bf03e0d7cc8dba73151dc1360c81a0689" exitCode=0 Jul 08 11:24:29.991335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.991276 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerDied","Data":"2f29e475e84ade6ebedca001278e0c0bf03e0d7cc8dba73151dc1360c81a0689"} Jul 08 11:24:29.991335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:29.991312 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerStarted","Data":"0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e"} Jul 08 11:24:30.016033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.015995 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn"] Jul 08 11:24:30.019606 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.019588 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.028986 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.028962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn"] Jul 08 11:24:30.088273 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.088219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.088273 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.088274 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.088475 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.088338 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4pct\" (UniqueName: \"kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.189260 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.189224 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.189260 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.189263 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.189463 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.189292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4pct\" (UniqueName: \"kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.189733 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.189710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.189776 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.189721 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.197549 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.197519 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4pct\" (UniqueName: \"kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.335994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.335907 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:30.419985 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.417935 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4"] Jul 08 11:24:30.422916 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.422888 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.431086 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.431060 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4"] Jul 08 11:24:30.495293 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.495256 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.495464 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.495311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4txt8\" (UniqueName: \"kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.495464 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.495448 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.496169 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.496151 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn"] Jul 08 11:24:30.498258 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:30.498232 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a62727_1d90_4b83_b75b_ff5a98957127.slice/crio-1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04 WatchSource:0}: Error finding container 1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04: Status 404 returned error can't find the container with id 1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04 Jul 08 11:24:30.596145 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.596050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.596145 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.596093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4txt8\" (UniqueName: \"kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.596514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.596163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.596584 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.596548 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.596584 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.596561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.605034 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.605011 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4txt8\" (UniqueName: \"kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.733380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.733347 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:30.809229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.809198 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8"] Jul 08 11:24:30.812919 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.812897 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:30.822557 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.822494 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8"] Jul 08 11:24:30.870033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.870007 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4"] Jul 08 11:24:30.887010 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:30.886969 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cef19d6_a3b6_44b8_867a_b29147720472.slice/crio-f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7 WatchSource:0}: Error finding container f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7: Status 404 returned error can't find the container with id f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7 Jul 08 11:24:30.899275 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.899251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:30.899406 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.899278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:30.899406 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.899305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2x8\" (UniqueName: \"kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:30.999177 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.999116 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerStarted","Data":"eb2b8b380b247bb1ece865fddb213427e7ab279cea28300c6ed836c15738589b"} Jul 08 11:24:30.999850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.999828 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:30.999850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.999862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.000090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:30.999903 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2x8\" (UniqueName: \"kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.000282 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.000256 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.000381 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.000309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.001158 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.001132 2559 generic.go:356] "Generic (PLEG): container finished" podID="88a62727-1d90-4b83-b75b-ff5a98957127" containerID="a6347b619d56dc635fef3c2cb9913bf6baa703e4875fd9c972d445ac9e6da2ab" exitCode=0 Jul 08 11:24:31.001237 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.001218 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" event={"ID":"88a62727-1d90-4b83-b75b-ff5a98957127","Type":"ContainerDied","Data":"a6347b619d56dc635fef3c2cb9913bf6baa703e4875fd9c972d445ac9e6da2ab"} Jul 08 11:24:31.001284 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.001260 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" event={"ID":"88a62727-1d90-4b83-b75b-ff5a98957127","Type":"ContainerStarted","Data":"1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04"} Jul 08 11:24:31.003027 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.003005 2559 generic.go:356] "Generic (PLEG): container finished" podID="4cef19d6-a3b6-44b8-867a-b29147720472" containerID="e9a56f051fb12b29d00b3b325ebbe8e1439f218ad672086ac64ae951bc3cd68c" exitCode=0 Jul 08 11:24:31.003137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.003089 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" event={"ID":"4cef19d6-a3b6-44b8-867a-b29147720472","Type":"ContainerDied","Data":"e9a56f051fb12b29d00b3b325ebbe8e1439f218ad672086ac64ae951bc3cd68c"} Jul 08 11:24:31.003186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.003140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" event={"ID":"4cef19d6-a3b6-44b8-867a-b29147720472","Type":"ContainerStarted","Data":"f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7"} Jul 08 11:24:31.008273 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.008248 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2x8\" (UniqueName: \"kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.127957 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.127865 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:31.260404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:31.260375 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8"] Jul 08 11:24:31.261822 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:31.261799 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5d5ca0_f052_42ca_a0dc_9571c0eeda2e.slice/crio-567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c WatchSource:0}: Error finding container 567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c: Status 404 returned error can't find the container with id 567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c Jul 08 11:24:32.014990 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.014952 2559 generic.go:356] "Generic (PLEG): container finished" podID="4cef19d6-a3b6-44b8-867a-b29147720472" containerID="41c2725852805a10e8c1c1c0ab369faaed4fb57682ec6a55d06a7c815970fb4b" exitCode=0 Jul 08 11:24:32.015419 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.015045 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" event={"ID":"4cef19d6-a3b6-44b8-867a-b29147720472","Type":"ContainerDied","Data":"41c2725852805a10e8c1c1c0ab369faaed4fb57682ec6a55d06a7c815970fb4b"} Jul 08 11:24:32.016618 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.016598 2559 generic.go:356] "Generic (PLEG): container finished" podID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerID="ba4b8fb4fbc9eac3883975133e9c0880d69ba5b6ddde3acfdf9548a18f3bc700" exitCode=0 Jul 08 11:24:32.016734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.016668 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" event={"ID":"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e","Type":"ContainerDied","Data":"ba4b8fb4fbc9eac3883975133e9c0880d69ba5b6ddde3acfdf9548a18f3bc700"} Jul 08 11:24:32.016734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.016713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" event={"ID":"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e","Type":"ContainerStarted","Data":"567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c"} Jul 08 11:24:32.018673 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.018655 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fc9a805-a906-471c-9573-9355c0fd9263" containerID="eb2b8b380b247bb1ece865fddb213427e7ab279cea28300c6ed836c15738589b" exitCode=0 Jul 08 11:24:32.018762 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:32.018696 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerDied","Data":"eb2b8b380b247bb1ece865fddb213427e7ab279cea28300c6ed836c15738589b"} Jul 08 11:24:33.025525 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.025488 2559 generic.go:356] "Generic (PLEG): container finished" podID="4cef19d6-a3b6-44b8-867a-b29147720472" containerID="2e39ef1275b1bbf9062832bb15470ac42c1e5c89abcc4e5399b18b69755ca04d" exitCode=0 Jul 08 11:24:33.025949 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.025538 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" event={"ID":"4cef19d6-a3b6-44b8-867a-b29147720472","Type":"ContainerDied","Data":"2e39ef1275b1bbf9062832bb15470ac42c1e5c89abcc4e5399b18b69755ca04d"} Jul 08 11:24:33.027670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.027622 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fc9a805-a906-471c-9573-9355c0fd9263" containerID="1bda535819db5e025ecab2ebf1b5fa79b8a711cfcfb429c9a96b892b89fec187" exitCode=0 Jul 08 11:24:33.027783 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.027666 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerDied","Data":"1bda535819db5e025ecab2ebf1b5fa79b8a711cfcfb429c9a96b892b89fec187"} Jul 08 11:24:33.029548 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.029526 2559 generic.go:356] "Generic (PLEG): container finished" podID="88a62727-1d90-4b83-b75b-ff5a98957127" containerID="2785563bb793aa22c202c7d7534fbca531a7f0427c9c6880b848d4d2de5001ac" exitCode=0 Jul 08 11:24:33.029628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:33.029550 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" event={"ID":"88a62727-1d90-4b83-b75b-ff5a98957127","Type":"ContainerDied","Data":"2785563bb793aa22c202c7d7534fbca531a7f0427c9c6880b848d4d2de5001ac"} Jul 08 11:24:34.036271 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.036241 2559 generic.go:356] "Generic (PLEG): container finished" podID="88a62727-1d90-4b83-b75b-ff5a98957127" containerID="eaab51e37405efb38da067f5fc675eb8c38ce23b2dad1af0ffa1ee871730c7e3" exitCode=0 Jul 08 11:24:34.036716 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.036327 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" event={"ID":"88a62727-1d90-4b83-b75b-ff5a98957127","Type":"ContainerDied","Data":"eaab51e37405efb38da067f5fc675eb8c38ce23b2dad1af0ffa1ee871730c7e3"} Jul 08 11:24:34.038112 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.038090 2559 generic.go:356] "Generic (PLEG): container finished" podID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerID="19bacda4f9be882f8c4de2551a28ce8230147b0684368f927401ebd6607deffa" exitCode=0 Jul 08 11:24:34.038205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.038157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" event={"ID":"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e","Type":"ContainerDied","Data":"19bacda4f9be882f8c4de2551a28ce8230147b0684368f927401ebd6607deffa"} Jul 08 11:24:34.241953 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.241926 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:34.248167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.248147 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:34.330649 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330543 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util\") pod \"8fc9a805-a906-471c-9573-9355c0fd9263\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " Jul 08 11:24:34.330649 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4txt8\" (UniqueName: \"kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8\") pod \"4cef19d6-a3b6-44b8-867a-b29147720472\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " Jul 08 11:24:34.330649 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330626 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs54c\" (UniqueName: \"kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c\") pod \"8fc9a805-a906-471c-9573-9355c0fd9263\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " Jul 08 11:24:34.330913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330691 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util\") pod \"4cef19d6-a3b6-44b8-867a-b29147720472\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " Jul 08 11:24:34.330913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330739 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle\") pod \"8fc9a805-a906-471c-9573-9355c0fd9263\" (UID: \"8fc9a805-a906-471c-9573-9355c0fd9263\") " Jul 08 11:24:34.330913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.330766 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle\") pod \"4cef19d6-a3b6-44b8-867a-b29147720472\" (UID: \"4cef19d6-a3b6-44b8-867a-b29147720472\") " Jul 08 11:24:34.331890 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.331727 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle" (OuterVolumeSpecName: "bundle") pod "4cef19d6-a3b6-44b8-867a-b29147720472" (UID: "4cef19d6-a3b6-44b8-867a-b29147720472"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:34.331890 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.331726 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle" (OuterVolumeSpecName: "bundle") pod "8fc9a805-a906-471c-9573-9355c0fd9263" (UID: "8fc9a805-a906-471c-9573-9355c0fd9263"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:34.333535 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.333472 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c" (OuterVolumeSpecName: "kube-api-access-gs54c") pod "8fc9a805-a906-471c-9573-9355c0fd9263" (UID: "8fc9a805-a906-471c-9573-9355c0fd9263"). InnerVolumeSpecName "kube-api-access-gs54c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:24:34.333694 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.333660 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8" (OuterVolumeSpecName: "kube-api-access-4txt8") pod "4cef19d6-a3b6-44b8-867a-b29147720472" (UID: "4cef19d6-a3b6-44b8-867a-b29147720472"). InnerVolumeSpecName "kube-api-access-4txt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:24:34.339210 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.339170 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util" (OuterVolumeSpecName: "util") pod "4cef19d6-a3b6-44b8-867a-b29147720472" (UID: "4cef19d6-a3b6-44b8-867a-b29147720472"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:34.339524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.339501 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util" (OuterVolumeSpecName: "util") pod "8fc9a805-a906-471c-9573-9355c0fd9263" (UID: "8fc9a805-a906-471c-9573-9355c0fd9263"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:34.431565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431525 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4txt8\" (UniqueName: \"kubernetes.io/projected/4cef19d6-a3b6-44b8-867a-b29147720472-kube-api-access-4txt8\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:34.431565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431559 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs54c\" (UniqueName: \"kubernetes.io/projected/8fc9a805-a906-471c-9573-9355c0fd9263-kube-api-access-gs54c\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:34.431565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431573 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:34.431833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431586 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:34.431833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431597 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cef19d6-a3b6-44b8-867a-b29147720472-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:34.431833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:34.431607 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8fc9a805-a906-471c-9573-9355c0fd9263-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:35.045041 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.045010 2559 generic.go:356] "Generic (PLEG): container finished" podID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerID="90ac6a123855524eb0502196bdf9cedb118e2c5e916b733c352968135287a417" exitCode=0 Jul 08 11:24:35.045416 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.045086 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" event={"ID":"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e","Type":"ContainerDied","Data":"90ac6a123855524eb0502196bdf9cedb118e2c5e916b733c352968135287a417"} Jul 08 11:24:35.046927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.046912 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" Jul 08 11:24:35.046997 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.046951 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp" event={"ID":"8fc9a805-a906-471c-9573-9355c0fd9263","Type":"ContainerDied","Data":"0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e"} Jul 08 11:24:35.047053 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.046998 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0421e63164a6984c4bb7484def2572d2e610a37e646405c71ebe2d58df5dea8e" Jul 08 11:24:35.048974 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.048956 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" Jul 08 11:24:35.049085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.048998 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4" event={"ID":"4cef19d6-a3b6-44b8-867a-b29147720472","Type":"ContainerDied","Data":"f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7"} Jul 08 11:24:35.049085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.049035 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f31eaba3956cd2c38fe9e98b2c288128e415277ea466de0702fdcd7232cc85e7" Jul 08 11:24:35.183753 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.183731 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:35.240368 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.240334 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4pct\" (UniqueName: \"kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct\") pod \"88a62727-1d90-4b83-b75b-ff5a98957127\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " Jul 08 11:24:35.240550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.240384 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util\") pod \"88a62727-1d90-4b83-b75b-ff5a98957127\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " Jul 08 11:24:35.240550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.240466 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle\") pod \"88a62727-1d90-4b83-b75b-ff5a98957127\" (UID: \"88a62727-1d90-4b83-b75b-ff5a98957127\") " Jul 08 11:24:35.241049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.241025 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle" (OuterVolumeSpecName: "bundle") pod "88a62727-1d90-4b83-b75b-ff5a98957127" (UID: "88a62727-1d90-4b83-b75b-ff5a98957127"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:35.242567 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.242546 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct" (OuterVolumeSpecName: "kube-api-access-x4pct") pod "88a62727-1d90-4b83-b75b-ff5a98957127" (UID: "88a62727-1d90-4b83-b75b-ff5a98957127"). InnerVolumeSpecName "kube-api-access-x4pct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:24:35.245936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.245902 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util" (OuterVolumeSpecName: "util") pod "88a62727-1d90-4b83-b75b-ff5a98957127" (UID: "88a62727-1d90-4b83-b75b-ff5a98957127"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:35.341326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.341239 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4pct\" (UniqueName: \"kubernetes.io/projected/88a62727-1d90-4b83-b75b-ff5a98957127-kube-api-access-x4pct\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:35.341326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.341276 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:35.341326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:35.341285 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88a62727-1d90-4b83-b75b-ff5a98957127-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:36.056080 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.056045 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" Jul 08 11:24:36.056080 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.056056 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn" event={"ID":"88a62727-1d90-4b83-b75b-ff5a98957127","Type":"ContainerDied","Data":"1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04"} Jul 08 11:24:36.056522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.056101 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d5720ca293affb9c8e5102578e056581d22dd699a2f90440e9fae94a3200b04" Jul 08 11:24:36.187878 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.187848 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:36.249236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.249199 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util\") pod \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " Jul 08 11:24:36.249236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.249241 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r2x8\" (UniqueName: \"kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8\") pod \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " Jul 08 11:24:36.249500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.249309 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle\") pod \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\" (UID: \"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e\") " Jul 08 11:24:36.249773 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.249750 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle" (OuterVolumeSpecName: "bundle") pod "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" (UID: "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:36.251516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.251496 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8" (OuterVolumeSpecName: "kube-api-access-6r2x8") pod "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" (UID: "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e"). InnerVolumeSpecName "kube-api-access-6r2x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:24:36.254492 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.254467 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util" (OuterVolumeSpecName: "util") pod "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" (UID: "5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:24:36.350354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.350265 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-bundle\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:36.350354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.350302 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-util\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:36.350354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:36.350318 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r2x8\" (UniqueName: \"kubernetes.io/projected/5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e-kube-api-access-6r2x8\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:24:37.062994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:37.062957 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" event={"ID":"5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e","Type":"ContainerDied","Data":"567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c"} Jul 08 11:24:37.063381 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:37.063031 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="567c3feba8f371f1fa6172d9ea4838fbf05b5e2c2f3c4d2f2ce13ac4b909f79c" Jul 08 11:24:37.063381 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:37.062976 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8" Jul 08 11:24:42.170968 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:42.170935 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:42.783380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:42.783345 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:47.192205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:47.192173 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:49.582963 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:49.582926 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:49.679184 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:49.679143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:52.116590 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:52.116528 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:52.407806 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:52.407719 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:53.655293 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:53.655246 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:53.896696 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:53.896624 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:54.400363 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400325 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d"] Jul 08 11:24:54.400665 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400652 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400666 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400683 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="pull" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400689 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="pull" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400696 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400702 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400707 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400712 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="util" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400717 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="extract" Jul 08 11:24:54.400718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400722 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400732 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="util" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400737 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="util" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400742 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400747 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400753 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400757 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400762 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400766 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400771 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400776 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="pull" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400783 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400788 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400793 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400797 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400847 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a62727-1d90-4b83-b75b-ff5a98957127" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400855 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cef19d6-a3b6-44b8-867a-b29147720472" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400863 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e" containerName="extract" Jul 08 11:24:54.401020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.400868 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc9a805-a906-471c-9573-9355c0fd9263" containerName="extract" Jul 08 11:24:54.403976 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.403959 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:54.406397 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.406368 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 11:24:54.406949 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.406932 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 11:24:54.407458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.407442 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 11:24:54.407506 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.407442 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hlmxh\"" Jul 08 11:24:54.414572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.414553 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d"] Jul 08 11:24:54.506038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.506001 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mtsr\" (UniqueName: \"kubernetes.io/projected/3104da5f-fe19-4ea5-ace1-f1930cfc54ea-kube-api-access-2mtsr\") pod \"dns-operator-controller-manager-864dd78c4c-7nl7d\" (UID: \"3104da5f-fe19-4ea5-ace1-f1930cfc54ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:54.606678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.606615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mtsr\" (UniqueName: \"kubernetes.io/projected/3104da5f-fe19-4ea5-ace1-f1930cfc54ea-kube-api-access-2mtsr\") pod \"dns-operator-controller-manager-864dd78c4c-7nl7d\" (UID: \"3104da5f-fe19-4ea5-ace1-f1930cfc54ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:54.618389 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.618353 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mtsr\" (UniqueName: \"kubernetes.io/projected/3104da5f-fe19-4ea5-ace1-f1930cfc54ea-kube-api-access-2mtsr\") pod \"dns-operator-controller-manager-864dd78c4c-7nl7d\" (UID: \"3104da5f-fe19-4ea5-ace1-f1930cfc54ea\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:54.714231 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.714197 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:54.847890 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.847864 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d"] Jul 08 11:24:54.849577 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:54.849553 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3104da5f_fe19_4ea5_ace1_f1930cfc54ea.slice/crio-fab484d612a0d4ca4183fe13afe380ea282390d47dfb25f54e960874e05069e9 WatchSource:0}: Error finding container fab484d612a0d4ca4183fe13afe380ea282390d47dfb25f54e960874e05069e9: Status 404 returned error can't find the container with id fab484d612a0d4ca4183fe13afe380ea282390d47dfb25f54e960874e05069e9 Jul 08 11:24:54.989178 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:54.989057 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:55.152171 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:55.152137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" event={"ID":"3104da5f-fe19-4ea5-ace1-f1930cfc54ea","Type":"ContainerStarted","Data":"fab484d612a0d4ca4183fe13afe380ea282390d47dfb25f54e960874e05069e9"} Jul 08 11:24:56.393519 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.393479 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:56.816467 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.816427 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq"] Jul 08 11:24:56.820299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.820276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:24:56.822872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.822852 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qkq4r\"" Jul 08 11:24:56.830848 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.830821 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq"] Jul 08 11:24:56.929253 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:56.929221 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzrdb\" (UniqueName: \"kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb\") pod \"limitador-operator-controller-manager-859d7b78bd-qcppq\" (UID: \"a81afa41-6549-46a9-8544-0ecfd385f0b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:24:57.030480 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:57.030431 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzrdb\" (UniqueName: \"kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb\") pod \"limitador-operator-controller-manager-859d7b78bd-qcppq\" (UID: \"a81afa41-6549-46a9-8544-0ecfd385f0b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:24:57.039896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:57.039856 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzrdb\" (UniqueName: \"kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb\") pod \"limitador-operator-controller-manager-859d7b78bd-qcppq\" (UID: \"a81afa41-6549-46a9-8544-0ecfd385f0b2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:24:57.134435 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:57.134364 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:24:57.282596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:57.282572 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq"] Jul 08 11:24:57.285125 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:24:57.285096 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81afa41_6549_46a9_8544_0ecfd385f0b2.slice/crio-fdea214248a3e9f5f7ed45f67ac394cc7835d232c7605fc208048d13c71d9515 WatchSource:0}: Error finding container fdea214248a3e9f5f7ed45f67ac394cc7835d232c7605fc208048d13c71d9515: Status 404 returned error can't find the container with id fdea214248a3e9f5f7ed45f67ac394cc7835d232c7605fc208048d13c71d9515 Jul 08 11:24:58.033524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.033490 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:58.169714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.169675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" event={"ID":"a81afa41-6549-46a9-8544-0ecfd385f0b2","Type":"ContainerStarted","Data":"fdea214248a3e9f5f7ed45f67ac394cc7835d232c7605fc208048d13c71d9515"} Jul 08 11:24:58.171495 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.171465 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" event={"ID":"3104da5f-fe19-4ea5-ace1-f1930cfc54ea","Type":"ContainerStarted","Data":"49a30df828486752b89ab2402343b0f7e88b296b023f75146baf55ba031f8adb"} Jul 08 11:24:58.171617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.171539 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:24:58.192254 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.192186 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" podStartSLOduration=1.896817975 podStartE2EDuration="4.192164181s" totalImagesPullingTime="2.295346206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:24:58.189222341 +0000 UTC m=+714.570869152" watchObservedRunningTime="2026-07-08 11:24:58.192164181 +0000 UTC m=+714.573810954" Jul 08 11:24:58.784490 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.784453 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:58.884982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:58.884947 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:59.184626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:59.184545 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:24:59.284869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:24:59.284837 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:00.183004 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:00.182962 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" event={"ID":"a81afa41-6549-46a9-8544-0ecfd385f0b2","Type":"ContainerStarted","Data":"74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0"} Jul 08 11:25:00.183188 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:00.183060 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:00.199680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:00.199605 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" podStartSLOduration=2.274589567 podStartE2EDuration="4.199587316s" totalImagesPullingTime="1.924997749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:25:00.198477875 +0000 UTC m=+716.580124648" watchObservedRunningTime="2026-07-08 11:25:00.199587316 +0000 UTC m=+716.581234087" Jul 08 11:25:00.687229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:00.687189 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:01.157512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:01.157474 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:02.038162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:02.038125 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:02.230422 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:02.230380 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:02.584623 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:02.584587 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:02.652493 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:02.652460 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:09.179290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:09.179259 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nl7d" Jul 08 11:25:10.456339 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:10.456299 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:10.626119 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:10.626085 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:11.193411 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.193381 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:11.312582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.312548 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq"] Jul 08 11:25:11.312797 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.312777 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" containerName="manager" containerID="cri-o://74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0" gracePeriod=2 Jul 08 11:25:11.322204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.322167 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq"] Jul 08 11:25:11.328555 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:25:11.328525 2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a81afa41-6549-46a9-8544-0ecfd385f0b2\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-qcppq\": pods \"limitador-operator-controller-manager-859d7b78bd-qcppq\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:11.555658 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.555611 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:11.563117 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:25:11.563087 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qcppq\" is forbidden: User \"system:node:ip-10-0-140-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-148.ec2.internal' and this object" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:11.592745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.592713 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:25:11.593062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.593050 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" containerName="manager" Jul 08 11:25:11.593104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.593063 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" containerName="manager" Jul 08 11:25:11.593141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.593126 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" containerName="manager" Jul 08 11:25:11.596104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.596090 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.598995 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.598966 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-trpgp\"" Jul 08 11:25:11.599141 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:25:11.599077 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qcppq\" is forbidden: User \"system:node:ip-10-0-140-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-148.ec2.internal' and this object" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:11.622235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.622206 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:25:11.647095 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.647062 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzrdb\" (UniqueName: \"kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb\") pod \"a81afa41-6549-46a9-8544-0ecfd385f0b2\" (UID: \"a81afa41-6549-46a9-8544-0ecfd385f0b2\") " Jul 08 11:25:11.647266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.647246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6w87\" (UniqueName: \"kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.647388 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.647371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.649257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.649239 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb" (OuterVolumeSpecName: "kube-api-access-tzrdb") pod "a81afa41-6549-46a9-8544-0ecfd385f0b2" (UID: "a81afa41-6549-46a9-8544-0ecfd385f0b2"). InnerVolumeSpecName "kube-api-access-tzrdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:25:11.748678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.748556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.748678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.748627 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6w87\" (UniqueName: \"kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.748870 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.748728 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzrdb\" (UniqueName: \"kubernetes.io/projected/a81afa41-6549-46a9-8544-0ecfd385f0b2-kube-api-access-tzrdb\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:25:11.748981 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.748962 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.765804 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.765779 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6w87\" (UniqueName: \"kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87\") pod \"kuadrant-operator-controller-manager-86878c769f-lt27m\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:11.906089 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:11.906039 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:12.039726 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.039702 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:25:12.041175 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:25:12.041151 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278e12d0_7cf0_4bab_9cd8_ac0ff4fb287c.slice/crio-25262b37d5e78fd961331a8dc9afaa4f9389755dc9fa5de14a4c47bd16534990 WatchSource:0}: Error finding container 25262b37d5e78fd961331a8dc9afaa4f9389755dc9fa5de14a4c47bd16534990: Status 404 returned error can't find the container with id 25262b37d5e78fd961331a8dc9afaa4f9389755dc9fa5de14a4c47bd16534990 Jul 08 11:25:12.168735 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.168702 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81afa41-6549-46a9-8544-0ecfd385f0b2" path="/var/lib/kubelet/pods/a81afa41-6549-46a9-8544-0ecfd385f0b2/volumes" Jul 08 11:25:12.247354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.247313 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" event={"ID":"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c","Type":"ContainerStarted","Data":"25262b37d5e78fd961331a8dc9afaa4f9389755dc9fa5de14a4c47bd16534990"} Jul 08 11:25:12.248536 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.248515 2559 generic.go:356] "Generic (PLEG): container finished" podID="a81afa41-6549-46a9-8544-0ecfd385f0b2" containerID="74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0" exitCode=0 Jul 08 11:25:12.248692 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.248557 2559 scope.go:117] "RemoveContainer" containerID="74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0" Jul 08 11:25:12.248692 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.248569 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qcppq" Jul 08 11:25:12.257655 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.257617 2559 scope.go:117] "RemoveContainer" containerID="74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0" Jul 08 11:25:12.257905 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:25:12.257886 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0\": container with ID starting with 74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0 not found: ID does not exist" containerID="74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0" Jul 08 11:25:12.257990 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:12.257916 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0"} err="failed to get container status \"74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0\": rpc error: code = NotFound desc = could not find container \"74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0\": container with ID starting with 74114fab770af9e5ca5057d435f2419777ba90753f36d930cf0be94ca32381e0 not found: ID does not exist" Jul 08 11:25:14.224492 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:14.224458 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:14.324050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:14.324012 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:17.282331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:17.282296 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" event={"ID":"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c","Type":"ContainerStarted","Data":"7abb7d4205c66233fbad9d6a65018da7246895af1e194a059e4d79c940e42081"} Jul 08 11:25:17.282770 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:17.282430 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:17.303114 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:17.303060 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" podStartSLOduration=1.5236976869999999 podStartE2EDuration="6.303045699s" totalImagesPullingTime="4.779348012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:25:17.301091782 +0000 UTC m=+733.682738553" watchObservedRunningTime="2026-07-08 11:25:17.303045699 +0000 UTC m=+733.684692469" Jul 08 11:25:17.992393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:17.992362 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:18.093877 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:18.093844 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:18.199533 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:18.199496 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:18.260748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:18.260658 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:20.685694 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:20.685658 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:20.785455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:20.785417 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:24.103427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:24.103393 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:27.429533 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:27.429497 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:27.489454 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:27.489421 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:28.290076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:28.290049 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:25:31.902553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:31.902516 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:32.003560 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:32.003533 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:35.703594 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:35.703559 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:35.753940 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:35.753911 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:36.097426 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:36.097391 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:36.152585 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:36.152538 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:40.098368 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:40.098332 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:40.163787 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:40.163748 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:44.716612 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.716576 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x"] Jul 08 11:25:44.722201 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.722177 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.724765 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.724740 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hktqt\"" Jul 08 11:25:44.731735 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.731695 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x"] Jul 08 11:25:44.738336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.738309 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:44.843852 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.843813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.843860 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.843891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.843966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.844019 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.844069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.844111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.844139 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.844325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.844162 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmfff\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-kube-api-access-wmfff\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945155 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945117 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945160 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945217 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945295 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945346 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmfff\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-kube-api-access-wmfff\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945385 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945596 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945674 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.945954 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.945882 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.946600 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.946573 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.947997 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.947977 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.948184 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.948163 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.953249 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.953225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmfff\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-kube-api-access-wmfff\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:44.953331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:44.953258 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vdk7x\" (UID: \"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:45.033919 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.033826 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:45.211750 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.211720 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x"] Jul 08 11:25:45.213094 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:25:45.213064 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162da7b4_b5e9_4a65_aaf0_e98d4dc6ed33.slice/crio-28f6b064be4465a8b2b5e388a597492acc1877b570cb1de19ee1c7ccf663ecbe WatchSource:0}: Error finding container 28f6b064be4465a8b2b5e388a597492acc1877b570cb1de19ee1c7ccf663ecbe: Status 404 returned error can't find the container with id 28f6b064be4465a8b2b5e388a597492acc1877b570cb1de19ee1c7ccf663ecbe Jul 08 11:25:45.215192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.215162 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:25:45.215281 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.215243 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:25:45.215344 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.215294 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:25:45.329479 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.329385 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:45.431541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.431510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" event={"ID":"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33","Type":"ContainerStarted","Data":"ef037219c1c3520ff56d87622f1df32bba08b7d967ef6bd2b51585abab36f0b1"} Jul 08 11:25:45.431731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.431559 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" event={"ID":"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33","Type":"ContainerStarted","Data":"28f6b064be4465a8b2b5e388a597492acc1877b570cb1de19ee1c7ccf663ecbe"} Jul 08 11:25:45.451000 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:45.450930 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" podStartSLOduration=1.450914453 podStartE2EDuration="1.450914453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:25:45.449234871 +0000 UTC m=+761.830881651" watchObservedRunningTime="2026-07-08 11:25:45.450914453 +0000 UTC m=+761.832561223" Jul 08 11:25:46.034619 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.034587 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:46.039509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.039486 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:46.436852 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.436762 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:46.437714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.437688 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:25:46.449524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.449492 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:46.556935 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.556879 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:46.613148 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:46.613119 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:47.256558 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.256525 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:47.260285 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.260261 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:47.262540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.262520 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lpkf8\"" Jul 08 11:25:47.270152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.270130 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:47.294476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.294448 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:47.369011 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.368979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj9pg\" (UniqueName: \"kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg\") pod \"authorino-58cd955675-jjmz2\" (UID: \"e1c354b6-3ec3-420b-bd28-a9602135eecf\") " pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:47.393926 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.393887 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:47.469518 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.469489 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj9pg\" (UniqueName: \"kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg\") pod \"authorino-58cd955675-jjmz2\" (UID: \"e1c354b6-3ec3-420b-bd28-a9602135eecf\") " pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:47.480833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.480802 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj9pg\" (UniqueName: \"kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg\") pod \"authorino-58cd955675-jjmz2\" (UID: \"e1c354b6-3ec3-420b-bd28-a9602135eecf\") " pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:47.570065 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.569976 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:47.708682 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:47.708658 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:47.710323 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:25:47.710295 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1c354b6_3ec3_420b_bd28_a9602135eecf.slice/crio-b45794978c29b69cc1efeab3962d5b097e2c3b7d97d851eb1c196cb8dc024e93 WatchSource:0}: Error finding container b45794978c29b69cc1efeab3962d5b097e2c3b7d97d851eb1c196cb8dc024e93: Status 404 returned error can't find the container with id b45794978c29b69cc1efeab3962d5b097e2c3b7d97d851eb1c196cb8dc024e93 Jul 08 11:25:48.102792 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:48.102752 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:48.168345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:48.168296 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:48.457231 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:48.456953 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jjmz2" event={"ID":"e1c354b6-3ec3-420b-bd28-a9602135eecf","Type":"ContainerStarted","Data":"b45794978c29b69cc1efeab3962d5b097e2c3b7d97d851eb1c196cb8dc024e93"} Jul 08 11:25:51.477430 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:51.477391 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jjmz2" event={"ID":"e1c354b6-3ec3-420b-bd28-a9602135eecf","Type":"ContainerStarted","Data":"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1"} Jul 08 11:25:51.493617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:51.493562 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-jjmz2" podStartSLOduration=1.197830563 podStartE2EDuration="4.493550034s" totalImagesPullingTime="3.295719471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:25:51.491518088 +0000 UTC m=+767.873164875" watchObservedRunningTime="2026-07-08 11:25:51.493550034 +0000 UTC m=+767.875196805" Jul 08 11:25:52.100271 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:52.100243 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:52.200569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:52.200540 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:25:53.897328 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:53.897290 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:53.897747 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:53.897473 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-jjmz2" podUID="e1c354b6-3ec3-420b-bd28-a9602135eecf" containerName="authorino" containerID="cri-o://0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1" gracePeriod=30 Jul 08 11:25:54.153835 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.153774 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:54.333031 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.332995 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj9pg\" (UniqueName: \"kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg\") pod \"e1c354b6-3ec3-420b-bd28-a9602135eecf\" (UID: \"e1c354b6-3ec3-420b-bd28-a9602135eecf\") " Jul 08 11:25:54.335078 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.335045 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg" (OuterVolumeSpecName: "kube-api-access-bj9pg") pod "e1c354b6-3ec3-420b-bd28-a9602135eecf" (UID: "e1c354b6-3ec3-420b-bd28-a9602135eecf"). InnerVolumeSpecName "kube-api-access-bj9pg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:25:54.433768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.433681 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj9pg\" (UniqueName: \"kubernetes.io/projected/e1c354b6-3ec3-420b-bd28-a9602135eecf-kube-api-access-bj9pg\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:25:54.492951 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.492915 2559 generic.go:356] "Generic (PLEG): container finished" podID="e1c354b6-3ec3-420b-bd28-a9602135eecf" containerID="0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1" exitCode=0 Jul 08 11:25:54.493146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.492959 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jjmz2" Jul 08 11:25:54.493146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.492991 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jjmz2" event={"ID":"e1c354b6-3ec3-420b-bd28-a9602135eecf","Type":"ContainerDied","Data":"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1"} Jul 08 11:25:54.493146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.493033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jjmz2" event={"ID":"e1c354b6-3ec3-420b-bd28-a9602135eecf","Type":"ContainerDied","Data":"b45794978c29b69cc1efeab3962d5b097e2c3b7d97d851eb1c196cb8dc024e93"} Jul 08 11:25:54.493146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.493050 2559 scope.go:117] "RemoveContainer" containerID="0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1" Jul 08 11:25:54.502924 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.502906 2559 scope.go:117] "RemoveContainer" containerID="0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1" Jul 08 11:25:54.503186 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:25:54.503166 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1\": container with ID starting with 0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1 not found: ID does not exist" containerID="0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1" Jul 08 11:25:54.503240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.503196 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1"} err="failed to get container status \"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1\": rpc error: code = NotFound desc = could not find container \"0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1\": container with ID starting with 0e43735d4cfc14002e947068fa5b1003887b5080e29761316bbe937d7604a1f1 not found: ID does not exist" Jul 08 11:25:54.514575 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.514540 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:54.523742 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:54.523712 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-jjmz2"] Jul 08 11:25:56.169381 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:25:56.169344 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1c354b6-3ec3-420b-bd28-a9602135eecf" path="/var/lib/kubelet/pods/e1c354b6-3ec3-420b-bd28-a9602135eecf/volumes" Jul 08 11:26:02.604424 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:02.604374 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:02.707106 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:02.707063 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:03.095011 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:03.094956 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:15.280079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.280041 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:15.518395 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.518362 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:15.518732 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.518719 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c354b6-3ec3-420b-bd28-a9602135eecf" containerName="authorino" Jul 08 11:26:15.518780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.518733 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c354b6-3ec3-420b-bd28-a9602135eecf" containerName="authorino" Jul 08 11:26:15.518819 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.518801 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c354b6-3ec3-420b-bd28-a9602135eecf" containerName="authorino" Jul 08 11:26:15.521314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.521283 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:15.523265 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.523249 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:15.525568 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.525544 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lpkf8\"" Jul 08 11:26:15.529105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.529083 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:15.603656 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.599724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znb7n\" (UniqueName: \"kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n\") pod \"authorino-5d4b67c446-w7rxd\" (UID: \"aa4281e0-71f6-4551-8efb-74e54f350b45\") " pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:15.700533 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.700493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znb7n\" (UniqueName: \"kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n\") pod \"authorino-5d4b67c446-w7rxd\" (UID: \"aa4281e0-71f6-4551-8efb-74e54f350b45\") " pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:15.708702 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.708674 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znb7n\" (UniqueName: \"kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n\") pod \"authorino-5d4b67c446-w7rxd\" (UID: \"aa4281e0-71f6-4551-8efb-74e54f350b45\") " pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:15.767088 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.767048 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:15.767352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.767339 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:15.794212 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.794148 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c598f576-z45mt"] Jul 08 11:26:15.798704 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.798683 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:15.804898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.804870 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c598f576-z45mt"] Jul 08 11:26:15.897241 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.897208 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c598f576-z45mt"] Jul 08 11:26:15.897491 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:26:15.897472 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rf4kg], unattached volumes=[], failed to process volumes=[kube-api-access-rf4kg]: context canceled" pod="kuadrant-system/authorino-5c598f576-z45mt" podUID="908a9156-a6b3-41f8-934d-cfc17073a60f" Jul 08 11:26:15.902207 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.902188 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf4kg\" (UniqueName: \"kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg\") pod \"authorino-5c598f576-z45mt\" (UID: \"908a9156-a6b3-41f8-934d-cfc17073a60f\") " pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:15.906399 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:15.906372 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:15.908015 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:15.907993 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa4281e0_71f6_4551_8efb_74e54f350b45.slice/crio-d7819cc6a56ae8b39c0e6951c39e7ad0f7d965bddf3ce92d716fed6de0de7b89 WatchSource:0}: Error finding container d7819cc6a56ae8b39c0e6951c39e7ad0f7d965bddf3ce92d716fed6de0de7b89: Status 404 returned error can't find the container with id d7819cc6a56ae8b39c0e6951c39e7ad0f7d965bddf3ce92d716fed6de0de7b89 Jul 08 11:26:16.002941 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.002898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf4kg\" (UniqueName: \"kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg\") pod \"authorino-5c598f576-z45mt\" (UID: \"908a9156-a6b3-41f8-934d-cfc17073a60f\") " pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:16.011709 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.011683 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf4kg\" (UniqueName: \"kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg\") pod \"authorino-5c598f576-z45mt\" (UID: \"908a9156-a6b3-41f8-934d-cfc17073a60f\") " pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:16.605858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.605774 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:16.605858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.605771 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" event={"ID":"aa4281e0-71f6-4551-8efb-74e54f350b45","Type":"ContainerStarted","Data":"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6"} Jul 08 11:26:16.605858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.605831 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" event={"ID":"aa4281e0-71f6-4551-8efb-74e54f350b45","Type":"ContainerStarted","Data":"d7819cc6a56ae8b39c0e6951c39e7ad0f7d965bddf3ce92d716fed6de0de7b89"} Jul 08 11:26:16.606374 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.605840 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" podUID="aa4281e0-71f6-4551-8efb-74e54f350b45" containerName="authorino" containerID="cri-o://dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6" gracePeriod=30 Jul 08 11:26:16.613907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.613884 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:16.621993 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.621951 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" podStartSLOduration=1.281973501 podStartE2EDuration="1.621937844s" totalImagesPullingTime="339.964343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:26:16.619669985 +0000 UTC m=+793.001316755" watchObservedRunningTime="2026-07-08 11:26:16.621937844 +0000 UTC m=+793.003584614" Jul 08 11:26:16.708385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.708350 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf4kg\" (UniqueName: \"kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg\") pod \"908a9156-a6b3-41f8-934d-cfc17073a60f\" (UID: \"908a9156-a6b3-41f8-934d-cfc17073a60f\") " Jul 08 11:26:16.710846 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.710812 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg" (OuterVolumeSpecName: "kube-api-access-rf4kg") pod "908a9156-a6b3-41f8-934d-cfc17073a60f" (UID: "908a9156-a6b3-41f8-934d-cfc17073a60f"). InnerVolumeSpecName "kube-api-access-rf4kg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:26:16.809393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.809356 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf4kg\" (UniqueName: \"kubernetes.io/projected/908a9156-a6b3-41f8-934d-cfc17073a60f-kube-api-access-rf4kg\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:26:16.850387 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.850364 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:16.909980 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.909894 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znb7n\" (UniqueName: \"kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n\") pod \"aa4281e0-71f6-4551-8efb-74e54f350b45\" (UID: \"aa4281e0-71f6-4551-8efb-74e54f350b45\") " Jul 08 11:26:16.912043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:16.912020 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n" (OuterVolumeSpecName: "kube-api-access-znb7n") pod "aa4281e0-71f6-4551-8efb-74e54f350b45" (UID: "aa4281e0-71f6-4551-8efb-74e54f350b45"). InnerVolumeSpecName "kube-api-access-znb7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:26:17.011278 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.011243 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znb7n\" (UniqueName: \"kubernetes.io/projected/aa4281e0-71f6-4551-8efb-74e54f350b45-kube-api-access-znb7n\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:26:17.612319 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612282 2559 generic.go:356] "Generic (PLEG): container finished" podID="aa4281e0-71f6-4551-8efb-74e54f350b45" containerID="dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6" exitCode=0 Jul 08 11:26:17.612827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612334 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" Jul 08 11:26:17.612827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" event={"ID":"aa4281e0-71f6-4551-8efb-74e54f350b45","Type":"ContainerDied","Data":"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6"} Jul 08 11:26:17.612827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612417 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-w7rxd" event={"ID":"aa4281e0-71f6-4551-8efb-74e54f350b45","Type":"ContainerDied","Data":"d7819cc6a56ae8b39c0e6951c39e7ad0f7d965bddf3ce92d716fed6de0de7b89"} Jul 08 11:26:17.612827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612442 2559 scope.go:117] "RemoveContainer" containerID="dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6" Jul 08 11:26:17.612827 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.612449 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c598f576-z45mt" Jul 08 11:26:17.621731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.621713 2559 scope.go:117] "RemoveContainer" containerID="dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6" Jul 08 11:26:17.621994 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:26:17.621972 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6\": container with ID starting with dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6 not found: ID does not exist" containerID="dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6" Jul 08 11:26:17.622053 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.622006 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6"} err="failed to get container status \"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6\": rpc error: code = NotFound desc = could not find container \"dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6\": container with ID starting with dfba077f222e98f9c26c0a376f93a0dbb3692239ffca3fc33b02aa4b3f5878c6 not found: ID does not exist" Jul 08 11:26:17.643024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.642997 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c598f576-z45mt"] Jul 08 11:26:17.654841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.654813 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c598f576-z45mt"] Jul 08 11:26:17.666758 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.666729 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:17.672914 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:17.672891 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w7rxd"] Jul 08 11:26:18.169146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:18.169105 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908a9156-a6b3-41f8-934d-cfc17073a60f" path="/var/lib/kubelet/pods/908a9156-a6b3-41f8-934d-cfc17073a60f/volumes" Jul 08 11:26:18.169423 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:18.169406 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4281e0-71f6-4551-8efb-74e54f350b45" path="/var/lib/kubelet/pods/aa4281e0-71f6-4551-8efb-74e54f350b45/volumes" Jul 08 11:26:21.337705 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.337669 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:21.361147 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.361118 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:21.361471 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.361457 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4281e0-71f6-4551-8efb-74e54f350b45" containerName="authorino" Jul 08 11:26:21.361534 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.361472 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4281e0-71f6-4551-8efb-74e54f350b45" containerName="authorino" Jul 08 11:26:21.361588 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.361547 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4281e0-71f6-4551-8efb-74e54f350b45" containerName="authorino" Jul 08 11:26:21.367743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.367717 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.370299 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.370274 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 11:26:21.370425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.370333 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j4clm\"" Jul 08 11:26:21.370425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.370333 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 11:26:21.378441 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.378414 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:21.436241 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.436205 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:21.450332 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.450300 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.450474 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.450364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkb9q\" (UniqueName: \"kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.490740 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.490711 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:21.550333 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.550298 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:26:21.551161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.551138 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkb9q\" (UniqueName: \"kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.551264 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.551191 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.553857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.553833 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.554773 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.554757 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.564800 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.564778 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkb9q\" (UniqueName: \"kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q\") pod \"maas-controller-bfbd564f7-k5d89\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.567412 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.567388 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:26:21.652326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.652235 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.652326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.652273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vgz9\" (UniqueName: \"kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.681769 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.681742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:21.753094 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.753064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.753269 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.753113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vgz9\" (UniqueName: \"kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.755558 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.755531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.762295 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.762271 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vgz9\" (UniqueName: \"kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9\") pod \"maas-controller-5b9657c48d-fltfv\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:21.817991 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.817962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:21.819210 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:21.819183 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731afbe2_3bc1_4b70_9397_4ea04edc7160.slice/crio-321eafee23399ba4953adaa8901e97798aee7d7ac620f0513e61e9bbedd0022d WatchSource:0}: Error finding container 321eafee23399ba4953adaa8901e97798aee7d7ac620f0513e61e9bbedd0022d: Status 404 returned error can't find the container with id 321eafee23399ba4953adaa8901e97798aee7d7ac620f0513e61e9bbedd0022d Jul 08 11:26:21.879387 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:21.879357 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:22.027227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:22.027199 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:26:22.028124 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:22.028094 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cb7c1a_4514_46fe_a0e3_58cbf26b7112.slice/crio-ac1145be4f1387882b70bcc7906f6d551834be4521747e1c21064f62642a3cd5 WatchSource:0}: Error finding container ac1145be4f1387882b70bcc7906f6d551834be4521747e1c21064f62642a3cd5: Status 404 returned error can't find the container with id ac1145be4f1387882b70bcc7906f6d551834be4521747e1c21064f62642a3cd5 Jul 08 11:26:22.648626 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:22.648587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-fltfv" event={"ID":"98cb7c1a-4514-46fe-a0e3-58cbf26b7112","Type":"ContainerStarted","Data":"ac1145be4f1387882b70bcc7906f6d551834be4521747e1c21064f62642a3cd5"} Jul 08 11:26:22.650686 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:22.650655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bfbd564f7-k5d89" event={"ID":"731afbe2-3bc1-4b70-9397-4ea04edc7160","Type":"ContainerStarted","Data":"321eafee23399ba4953adaa8901e97798aee7d7ac620f0513e61e9bbedd0022d"} Jul 08 11:26:24.230794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.230707 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:24.327060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.327025 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:24.663355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.663250 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-fltfv" event={"ID":"98cb7c1a-4514-46fe-a0e3-58cbf26b7112","Type":"ContainerStarted","Data":"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db"} Jul 08 11:26:24.663533 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.663444 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:24.664819 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.664795 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bfbd564f7-k5d89" event={"ID":"731afbe2-3bc1-4b70-9397-4ea04edc7160","Type":"ContainerStarted","Data":"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5"} Jul 08 11:26:24.664897 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.664840 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:24.682648 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.682590 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b9657c48d-fltfv" podStartSLOduration=1.7531487270000001 podStartE2EDuration="3.682578404s" totalImagesPullingTime="1.929429677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:26:24.681193495 +0000 UTC m=+801.062840295" watchObservedRunningTime="2026-07-08 11:26:24.682578404 +0000 UTC m=+801.064225175" Jul 08 11:26:24.698223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:24.698183 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bfbd564f7-k5d89" podStartSLOduration=1.55300222 podStartE2EDuration="3.698169872s" totalImagesPullingTime="2.145167652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:26:24.69691759 +0000 UTC m=+801.078564362" watchObservedRunningTime="2026-07-08 11:26:24.698169872 +0000 UTC m=+801.079816654" Jul 08 11:26:35.676925 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:35.676894 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:35.677409 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:35.676962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:26:35.735668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:35.735618 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:35.735853 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:35.735830 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-bfbd564f7-k5d89" podUID="731afbe2-3bc1-4b70-9397-4ea04edc7160" containerName="manager" containerID="cri-o://ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5" gracePeriod=30 Jul 08 11:26:35.989118 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:35.989093 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:36.081797 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.081768 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkb9q\" (UniqueName: \"kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q\") pod \"731afbe2-3bc1-4b70-9397-4ea04edc7160\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " Jul 08 11:26:36.081963 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.081851 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert\") pod \"731afbe2-3bc1-4b70-9397-4ea04edc7160\" (UID: \"731afbe2-3bc1-4b70-9397-4ea04edc7160\") " Jul 08 11:26:36.084134 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.084100 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "731afbe2-3bc1-4b70-9397-4ea04edc7160" (UID: "731afbe2-3bc1-4b70-9397-4ea04edc7160"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:26:36.084257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.084173 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q" (OuterVolumeSpecName: "kube-api-access-bkb9q") pod "731afbe2-3bc1-4b70-9397-4ea04edc7160" (UID: "731afbe2-3bc1-4b70-9397-4ea04edc7160"). InnerVolumeSpecName "kube-api-access-bkb9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:26:36.182487 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.182464 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkb9q\" (UniqueName: \"kubernetes.io/projected/731afbe2-3bc1-4b70-9397-4ea04edc7160-kube-api-access-bkb9q\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:26:36.182603 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.182505 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/731afbe2-3bc1-4b70-9397-4ea04edc7160-webhook-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:26:36.730715 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.730681 2559 generic.go:356] "Generic (PLEG): container finished" podID="731afbe2-3bc1-4b70-9397-4ea04edc7160" containerID="ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5" exitCode=0 Jul 08 11:26:36.731110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.730761 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bfbd564f7-k5d89" Jul 08 11:26:36.731110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.730767 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bfbd564f7-k5d89" event={"ID":"731afbe2-3bc1-4b70-9397-4ea04edc7160","Type":"ContainerDied","Data":"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5"} Jul 08 11:26:36.731110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.730859 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bfbd564f7-k5d89" event={"ID":"731afbe2-3bc1-4b70-9397-4ea04edc7160","Type":"ContainerDied","Data":"321eafee23399ba4953adaa8901e97798aee7d7ac620f0513e61e9bbedd0022d"} Jul 08 11:26:36.731110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.730891 2559 scope.go:117] "RemoveContainer" containerID="ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5" Jul 08 11:26:36.740103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.739912 2559 scope.go:117] "RemoveContainer" containerID="ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5" Jul 08 11:26:36.740208 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:26:36.740185 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5\": container with ID starting with ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5 not found: ID does not exist" containerID="ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5" Jul 08 11:26:36.740256 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.740220 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5"} err="failed to get container status \"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5\": rpc error: code = NotFound desc = could not find container \"ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5\": container with ID starting with ea941bedcaea5031fc282357a42d7264354de3b7c426edec4e14a458acc4f2c5 not found: ID does not exist" Jul 08 11:26:36.749229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.749205 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:36.752580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:36.752556 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-bfbd564f7-k5d89"] Jul 08 11:26:38.169481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:38.169446 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="731afbe2-3bc1-4b70-9397-4ea04edc7160" path="/var/lib/kubelet/pods/731afbe2-3bc1-4b70-9397-4ea04edc7160/volumes" Jul 08 11:26:38.391886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:38.391843 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:56.584614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:56.584573 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:57.028830 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.028780 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:57.102178 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.102138 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-69669b67-kpmw4"] Jul 08 11:26:57.102514 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.102502 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731afbe2-3bc1-4b70-9397-4ea04edc7160" containerName="manager" Jul 08 11:26:57.102557 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.102516 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="731afbe2-3bc1-4b70-9397-4ea04edc7160" containerName="manager" Jul 08 11:26:57.102611 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.102602 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="731afbe2-3bc1-4b70-9397-4ea04edc7160" containerName="manager" Jul 08 11:26:57.108425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.108401 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.110897 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.110873 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 08 11:26:57.111087 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.110905 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 11:26:57.111087 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.111006 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 11:26:57.112153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.112128 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tnv7f\"" Jul 08 11:26:57.114673 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.114649 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69669b67-kpmw4"] Jul 08 11:26:57.167473 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.167444 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:57.264060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.264021 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1f98e2-8975-4315-9abb-55ba21790026-openshift-service-ca\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.264244 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.264067 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd1f98e2-8975-4315-9abb-55ba21790026-maas-api-tls\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.264244 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.264156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frn2x\" (UniqueName: \"kubernetes.io/projected/cd1f98e2-8975-4315-9abb-55ba21790026-kube-api-access-frn2x\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.365365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.365276 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1f98e2-8975-4315-9abb-55ba21790026-openshift-service-ca\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.365365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.365318 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd1f98e2-8975-4315-9abb-55ba21790026-maas-api-tls\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.365365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.365362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frn2x\" (UniqueName: \"kubernetes.io/projected/cd1f98e2-8975-4315-9abb-55ba21790026-kube-api-access-frn2x\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.365943 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.365924 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd1f98e2-8975-4315-9abb-55ba21790026-openshift-service-ca\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.367925 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.367903 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd1f98e2-8975-4315-9abb-55ba21790026-maas-api-tls\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.373521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.373494 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frn2x\" (UniqueName: \"kubernetes.io/projected/cd1f98e2-8975-4315-9abb-55ba21790026-kube-api-access-frn2x\") pod \"maas-api-69669b67-kpmw4\" (UID: \"cd1f98e2-8975-4315-9abb-55ba21790026\") " pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.420427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.420388 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:26:57.464227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.464189 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:57.562836 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.562807 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:26:57.576875 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.576849 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69669b67-kpmw4"] Jul 08 11:26:57.578465 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:57.578438 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1f98e2_8975_4315_9abb_55ba21790026.slice/crio-783d3a02e7e33a966f70e00422b812cb5bfd0789138f3e5f7c1e058fbf865614 WatchSource:0}: Error finding container 783d3a02e7e33a966f70e00422b812cb5bfd0789138f3e5f7c1e058fbf865614: Status 404 returned error can't find the container with id 783d3a02e7e33a966f70e00422b812cb5bfd0789138f3e5f7c1e058fbf865614 Jul 08 11:26:57.579899 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.579879 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 11:26:57.608378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.608340 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-v5kzn"] Jul 08 11:26:57.614824 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.614806 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.617926 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.617862 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 11:26:57.617926 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.617908 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fpn89\"" Jul 08 11:26:57.622918 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.622890 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-v5kzn"] Jul 08 11:26:57.647447 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.647411 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-pph62"] Jul 08 11:26:57.652034 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.652015 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.660574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.660547 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-pph62"] Jul 08 11:26:57.768596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.768564 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwk5r\" (UniqueName: \"kubernetes.io/projected/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-kube-api-access-xwk5r\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.768826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.768610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-plugins-config-volume\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.768826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.768667 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a006a2d-051a-47a9-a46f-80fe737267ad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.768826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.768752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7rm\" (UniqueName: \"kubernetes.io/projected/3a006a2d-051a-47a9-a46f-80fe737267ad-kube-api-access-8x7rm\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.842133 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.842100 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" event={"ID":"cd1f98e2-8975-4315-9abb-55ba21790026","Type":"ContainerStarted","Data":"783d3a02e7e33a966f70e00422b812cb5bfd0789138f3e5f7c1e058fbf865614"} Jul 08 11:26:57.869900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.869804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a006a2d-051a-47a9-a46f-80fe737267ad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.869900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.869876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7rm\" (UniqueName: \"kubernetes.io/projected/3a006a2d-051a-47a9-a46f-80fe737267ad-kube-api-access-8x7rm\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.870146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.869933 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwk5r\" (UniqueName: \"kubernetes.io/projected/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-kube-api-access-xwk5r\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.870146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.869967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-plugins-config-volume\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.870621 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.870599 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a006a2d-051a-47a9-a46f-80fe737267ad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.870904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.870873 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-plugins-config-volume\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.878059 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.878037 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwk5r\" (UniqueName: \"kubernetes.io/projected/aaaa7957-7312-4181-8fd1-c3d6e8db4f19-kube-api-access-xwk5r\") pod \"payload-processing-58f86c7798-v5kzn\" (UID: \"aaaa7957-7312-4181-8fd1-c3d6e8db4f19\") " pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.878404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.878380 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7rm\" (UniqueName: \"kubernetes.io/projected/3a006a2d-051a-47a9-a46f-80fe737267ad-kube-api-access-8x7rm\") pod \"payload-pre-processing-7964457696-pph62\" (UID: \"3a006a2d-051a-47a9-a46f-80fe737267ad\") " pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:57.928037 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.928001 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:26:57.963384 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:57.963351 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:26:58.080156 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:58.080123 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-v5kzn"] Jul 08 11:26:58.081987 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:58.081958 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaaa7957_7312_4181_8fd1_c3d6e8db4f19.slice/crio-d4d3fc1544a3e9299a7c169d90d4b10905c513b25f81bfe4ab0f4dff0dbcd73c WatchSource:0}: Error finding container d4d3fc1544a3e9299a7c169d90d4b10905c513b25f81bfe4ab0f4dff0dbcd73c: Status 404 returned error can't find the container with id d4d3fc1544a3e9299a7c169d90d4b10905c513b25f81bfe4ab0f4dff0dbcd73c Jul 08 11:26:58.124085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:58.124060 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-pph62"] Jul 08 11:26:58.126412 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:26:58.126384 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a006a2d_051a_47a9_a46f_80fe737267ad.slice/crio-b2f1b9af6699c188d3cdfe70d93939fc0cfe9c6512e6cd4f198910b7557dbafd WatchSource:0}: Error finding container b2f1b9af6699c188d3cdfe70d93939fc0cfe9c6512e6cd4f198910b7557dbafd: Status 404 returned error can't find the container with id b2f1b9af6699c188d3cdfe70d93939fc0cfe9c6512e6cd4f198910b7557dbafd Jul 08 11:26:58.867523 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:58.867486 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" event={"ID":"3a006a2d-051a-47a9-a46f-80fe737267ad","Type":"ContainerStarted","Data":"b2f1b9af6699c188d3cdfe70d93939fc0cfe9c6512e6cd4f198910b7557dbafd"} Jul 08 11:26:58.869392 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:26:58.869365 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" event={"ID":"aaaa7957-7312-4181-8fd1-c3d6e8db4f19","Type":"ContainerStarted","Data":"d4d3fc1544a3e9299a7c169d90d4b10905c513b25f81bfe4ab0f4dff0dbcd73c"} Jul 08 11:27:00.883611 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.883577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" event={"ID":"cd1f98e2-8975-4315-9abb-55ba21790026","Type":"ContainerStarted","Data":"93a4db8d313474c2ab9a80d8fd37bc35f543532885d9af4062bce473d8c1c11d"} Jul 08 11:27:00.884359 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.883812 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:27:00.885433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.885402 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" event={"ID":"3a006a2d-051a-47a9-a46f-80fe737267ad","Type":"ContainerStarted","Data":"28d184ee1080bf70ce82bf9b06c347ea223534ed1f7db87ccc4f73a5a8710aaf"} Jul 08 11:27:00.885624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.885459 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:27:00.886962 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.886939 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" event={"ID":"aaaa7957-7312-4181-8fd1-c3d6e8db4f19","Type":"ContainerStarted","Data":"725ac892680b9ab92c3bd2fa65e3b35d3bc9e09e390c297b8bb56c16eaf7fb38"} Jul 08 11:27:00.887086 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.887071 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:27:00.904105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.904058 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" podStartSLOduration=1.251502611 podStartE2EDuration="3.904044378s" totalImagesPullingTime="2.652541767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:27:00.903245348 +0000 UTC m=+837.284892119" watchObservedRunningTime="2026-07-08 11:27:00.904044378 +0000 UTC m=+837.285691181" Jul 08 11:27:00.920303 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.920245 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" podStartSLOduration=1.753898271 podStartE2EDuration="3.920229156s" totalImagesPullingTime="2.166330885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:27:00.918267992 +0000 UTC m=+837.299914763" watchObservedRunningTime="2026-07-08 11:27:00.920229156 +0000 UTC m=+837.301875928" Jul 08 11:27:00.935871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:00.935806 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" podStartSLOduration=1.8317214000000002 podStartE2EDuration="3.935784765s" totalImagesPullingTime="2.104063365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:27:00.9337984 +0000 UTC m=+837.315445171" watchObservedRunningTime="2026-07-08 11:27:00.935784765 +0000 UTC m=+837.317431537" Jul 08 11:27:05.916121 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.916088 2559 generic.go:356] "Generic (PLEG): container finished" podID="162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33" containerID="ef037219c1c3520ff56d87622f1df32bba08b7d967ef6bd2b51585abab36f0b1" exitCode=0 Jul 08 11:27:05.916550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.916160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" event={"ID":"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33","Type":"ContainerDied","Data":"ef037219c1c3520ff56d87622f1df32bba08b7d967ef6bd2b51585abab36f0b1"} Jul 08 11:27:05.916593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.916564 2559 scope.go:117] "RemoveContainer" containerID="ef037219c1c3520ff56d87622f1df32bba08b7d967ef6bd2b51585abab36f0b1" Jul 08 11:27:05.917525 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.917500 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:27:05.917578 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.917569 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:27:05.917614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:05.917598 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:27:06.898699 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:06.898668 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-69669b67-kpmw4" Jul 08 11:27:06.925101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:06.925067 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" event={"ID":"162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33","Type":"ContainerStarted","Data":"0a7d9a684bd4ff5cce7c1c9df013e31271125d37c3885f52f91982234c579552"} Jul 08 11:27:07.034625 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:07.034591 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:27:07.034904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:07.034871 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vdk7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 11:27:07.034974 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:07.034930 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" podUID="162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 11:27:07.905371 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:07.905323 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:07.997280 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:07.997243 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:08.035220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:08.035175 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vdk7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 11:27:08.035419 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:08.035258 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" podUID="162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 11:27:09.034451 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:09.034409 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vdk7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 11:27:09.034927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:09.034475 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" podUID="162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 11:27:10.034368 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:10.034332 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vdk7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 08 11:27:10.034556 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:10.034391 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" podUID="162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 08 11:27:11.038074 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:11.038042 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:27:11.038544 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:11.038481 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:27:11.039118 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:11.039100 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vdk7x" Jul 08 11:27:11.894365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:11.894335 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-v5kzn" Jul 08 11:27:11.894538 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:11.894420 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-pph62" Jul 08 11:27:12.713979 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:12.713946 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:26.895460 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.895416 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr"] Jul 08 11:27:26.900289 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.900261 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:26.902685 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.902662 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5mvkj\"" Jul 08 11:27:26.902685 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.902670 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 11:27:26.903718 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.903700 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 11:27:26.903831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.903724 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 11:27:26.908869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.908842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr"] Jul 08 11:27:26.921058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:26.921026 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:27.037650 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.037850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab6a3133-9ed8-4053-8786-20abdef4e66e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.037850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037734 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89lh4\" (UniqueName: \"kubernetes.io/projected/ab6a3133-9ed8-4053-8786-20abdef4e66e-kube-api-access-89lh4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.037850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.037850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037837 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.038036 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.037878 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.138872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.138825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89lh4\" (UniqueName: \"kubernetes.io/projected/ab6a3133-9ed8-4053-8786-20abdef4e66e-kube-api-access-89lh4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.138880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.138905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.138934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.139037 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.139090 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab6a3133-9ed8-4053-8786-20abdef4e66e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139450 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.139428 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.139465 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.139532 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.139503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.141560 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.141532 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab6a3133-9ed8-4053-8786-20abdef4e66e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.141960 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.141936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab6a3133-9ed8-4053-8786-20abdef4e66e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.148724 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.148671 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89lh4\" (UniqueName: \"kubernetes.io/projected/ab6a3133-9ed8-4053-8786-20abdef4e66e-kube-api-access-89lh4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf7sr\" (UID: \"ab6a3133-9ed8-4053-8786-20abdef4e66e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.212291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.212248 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:27.370539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:27.370503 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr"] Jul 08 11:27:27.371579 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:27:27.371554 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6a3133_9ed8_4053_8786_20abdef4e66e.slice/crio-6fe46c018784304587d064287ec9675fd5ca8ad605a0b5bdc983f78035ebd23e WatchSource:0}: Error finding container 6fe46c018784304587d064287ec9675fd5ca8ad605a0b5bdc983f78035ebd23e: Status 404 returned error can't find the container with id 6fe46c018784304587d064287ec9675fd5ca8ad605a0b5bdc983f78035ebd23e Jul 08 11:27:28.049736 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:28.049679 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" event={"ID":"ab6a3133-9ed8-4053-8786-20abdef4e66e","Type":"ContainerStarted","Data":"6fe46c018784304587d064287ec9675fd5ca8ad605a0b5bdc983f78035ebd23e"} Jul 08 11:27:29.056924 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:29.056878 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" event={"ID":"ab6a3133-9ed8-4053-8786-20abdef4e66e","Type":"ContainerStarted","Data":"fbfb8278c1b42cc8aab3cf5ff1e237ab0ddf60c333df74e807ce37c3ff8b915f"} Jul 08 11:27:29.076943 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:29.076882 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" podStartSLOduration=2.018371606 podStartE2EDuration="3.076866553s" totalImagesPullingTime="1.058494947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:27:29.076226923 +0000 UTC m=+865.457873691" watchObservedRunningTime="2026-07-08 11:27:29.076866553 +0000 UTC m=+865.458513323" Jul 08 11:27:37.212591 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:37.212544 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:37.213025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:37.212623 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:37.225440 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:37.225407 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:38.114018 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:38.113992 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf7sr" Jul 08 11:27:40.271470 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:40.271434 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:44.828340 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:44.828306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:54.090298 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.090253 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz"] Jul 08 11:27:54.095425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.095401 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.097884 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.097862 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 11:27:54.103802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.103775 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz"] Jul 08 11:27:54.170186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170158 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.170316 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170299 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.170402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170383 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq4bh\" (UniqueName: \"kubernetes.io/projected/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-kube-api-access-qq4bh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.170462 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.170516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.170568 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.170516 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.189499 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.189468 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:54.271200 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271200 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271203 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271355 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271520 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq4bh\" (UniqueName: \"kubernetes.io/projected/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-kube-api-access-qq4bh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271659 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271682 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.271729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.271704 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.273762 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.273731 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.273912 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.273897 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.280227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.280204 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq4bh\" (UniqueName: \"kubernetes.io/projected/c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3-kube-api-access-qq4bh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c64vz\" (UID: \"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.426717 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.426580 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:27:54.569429 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:54.569392 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz"] Jul 08 11:27:54.570829 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:27:54.570801 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22f0dc8_83d5_48e2_90bb_7fa2c2fff7d3.slice/crio-0a4950afbee7e395f11949bddf80d6979b4820ced3907fa624c47f001e31154e WatchSource:0}: Error finding container 0a4950afbee7e395f11949bddf80d6979b4820ced3907fa624c47f001e31154e: Status 404 returned error can't find the container with id 0a4950afbee7e395f11949bddf80d6979b4820ced3907fa624c47f001e31154e Jul 08 11:27:55.190436 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:55.190310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" event={"ID":"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3","Type":"ContainerStarted","Data":"ff2f5a09ae7b5947d9fac0282e257fa29a2095cd6ed3f1f2ad84e49fc90be034"} Jul 08 11:27:55.190436 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:55.190401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" event={"ID":"c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3","Type":"ContainerStarted","Data":"0a4950afbee7e395f11949bddf80d6979b4820ced3907fa624c47f001e31154e"} Jul 08 11:27:56.008946 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:56.008910 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:27:59.301687 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:27:59.301650 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:28:04.108459 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.108432 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:28:04.109864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.109846 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:28:04.427187 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.427097 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:28:04.427187 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.427159 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:28:04.440895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.440861 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:28:04.460025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:04.459974 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" podStartSLOduration=10.273401014 podStartE2EDuration="10.459961239s" totalImagesPullingTime="186.560225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:27:55.206675245 +0000 UTC m=+891.588322015" watchObservedRunningTime="2026-07-08 11:28:04.459961239 +0000 UTC m=+900.841608048" Jul 08 11:28:05.262508 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:05.262478 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c64vz" Jul 08 11:28:36.490771 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:28:36.490736 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:03.148540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:03.148499 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:05.295584 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:05.295552 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:05.394106 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:05.394072 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:06.414439 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:06.414402 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:06.498828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:06.498780 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:08.027391 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:08.027348 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:29:08.127151 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:29:08.127111 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:30:26.537477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.537432 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:30:26.634033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.633980 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:30:26.634235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.634191 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5b9657c48d-fltfv" podUID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" containerName="manager" containerID="cri-o://25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db" gracePeriod=30 Jul 08 11:30:26.892670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.892627 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:30:26.934733 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.934697 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert\") pod \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " Jul 08 11:30:26.934909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.934837 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vgz9\" (UniqueName: \"kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9\") pod \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\" (UID: \"98cb7c1a-4514-46fe-a0e3-58cbf26b7112\") " Jul 08 11:30:26.937046 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.937007 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "98cb7c1a-4514-46fe-a0e3-58cbf26b7112" (UID: "98cb7c1a-4514-46fe-a0e3-58cbf26b7112"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:30:26.937481 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:26.937459 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9" (OuterVolumeSpecName: "kube-api-access-7vgz9") pod "98cb7c1a-4514-46fe-a0e3-58cbf26b7112" (UID: "98cb7c1a-4514-46fe-a0e3-58cbf26b7112"). InnerVolumeSpecName "kube-api-access-7vgz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:30:27.021206 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.021171 2559 generic.go:356] "Generic (PLEG): container finished" podID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" containerID="25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db" exitCode=0 Jul 08 11:30:27.021421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.021230 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-fltfv" Jul 08 11:30:27.021421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.021263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-fltfv" event={"ID":"98cb7c1a-4514-46fe-a0e3-58cbf26b7112","Type":"ContainerDied","Data":"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db"} Jul 08 11:30:27.021421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.021330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-fltfv" event={"ID":"98cb7c1a-4514-46fe-a0e3-58cbf26b7112","Type":"ContainerDied","Data":"ac1145be4f1387882b70bcc7906f6d551834be4521747e1c21064f62642a3cd5"} Jul 08 11:30:27.021421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.021360 2559 scope.go:117] "RemoveContainer" containerID="25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db" Jul 08 11:30:27.033211 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.033189 2559 scope.go:117] "RemoveContainer" containerID="25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db" Jul 08 11:30:27.033500 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:30:27.033477 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db\": container with ID starting with 25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db not found: ID does not exist" containerID="25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db" Jul 08 11:30:27.033604 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.033508 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db"} err="failed to get container status \"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db\": rpc error: code = NotFound desc = could not find container \"25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db\": container with ID starting with 25b2b51aac5d1bd53bbf7948ec0dbb3316f601babef1096c2ae8d9170486e5db not found: ID does not exist" Jul 08 11:30:27.035546 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.035526 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vgz9\" (UniqueName: \"kubernetes.io/projected/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-kube-api-access-7vgz9\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:30:27.035601 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.035552 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98cb7c1a-4514-46fe-a0e3-58cbf26b7112-webhook-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:30:27.039714 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:30:27.039693 2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cb7c1a_4514_46fe_a0e3_58cbf26b7112.slice\": RecentStats: unable to find data in memory cache]" Jul 08 11:30:27.045627 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.045600 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:30:27.047800 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.047780 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-fltfv"] Jul 08 11:30:27.797649 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.797612 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b9657c48d-mz6lj"] Jul 08 11:30:27.798030 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.798013 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" containerName="manager" Jul 08 11:30:27.798030 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.798024 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" containerName="manager" Jul 08 11:30:27.798110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.798084 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" containerName="manager" Jul 08 11:30:27.802422 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.802400 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.805959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.805939 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j4clm\"" Jul 08 11:30:27.806070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.805960 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 11:30:27.806070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.805939 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 11:30:27.812874 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.812854 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-mz6lj"] Jul 08 11:30:27.843688 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.843652 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm8xj\" (UniqueName: \"kubernetes.io/projected/18d25f9e-f54c-46da-9666-0ec88e48327f-kube-api-access-tm8xj\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.843845 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.843753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18d25f9e-f54c-46da-9666-0ec88e48327f-webhook-cert\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.945163 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.945122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8xj\" (UniqueName: \"kubernetes.io/projected/18d25f9e-f54c-46da-9666-0ec88e48327f-kube-api-access-tm8xj\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.945355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.945219 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18d25f9e-f54c-46da-9666-0ec88e48327f-webhook-cert\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.947860 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.947830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18d25f9e-f54c-46da-9666-0ec88e48327f-webhook-cert\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:27.953500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:27.953473 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm8xj\" (UniqueName: \"kubernetes.io/projected/18d25f9e-f54c-46da-9666-0ec88e48327f-kube-api-access-tm8xj\") pod \"maas-controller-5b9657c48d-mz6lj\" (UID: \"18d25f9e-f54c-46da-9666-0ec88e48327f\") " pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:28.022777 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:28.022736 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:30:28.113336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:28.113249 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:28.172757 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:28.172728 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98cb7c1a-4514-46fe-a0e3-58cbf26b7112" path="/var/lib/kubelet/pods/98cb7c1a-4514-46fe-a0e3-58cbf26b7112/volumes" Jul 08 11:30:28.255822 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:28.255796 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b9657c48d-mz6lj"] Jul 08 11:30:28.257159 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:30:28.257132 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d25f9e_f54c_46da_9666_0ec88e48327f.slice/crio-5e744c6dfb8eff3519c202c508b6e4602d5c1f1e17cfc18cb7a6953d77a548f2 WatchSource:0}: Error finding container 5e744c6dfb8eff3519c202c508b6e4602d5c1f1e17cfc18cb7a6953d77a548f2: Status 404 returned error can't find the container with id 5e744c6dfb8eff3519c202c508b6e4602d5c1f1e17cfc18cb7a6953d77a548f2 Jul 08 11:30:29.034317 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:29.034278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" event={"ID":"18d25f9e-f54c-46da-9666-0ec88e48327f","Type":"ContainerStarted","Data":"31cc763585399427bd0d8c379dcb43e85b19db13042d2eb7bb78bbd1a1873bd4"} Jul 08 11:30:29.034792 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:29.034488 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:29.034792 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:29.034522 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" event={"ID":"18d25f9e-f54c-46da-9666-0ec88e48327f","Type":"ContainerStarted","Data":"5e744c6dfb8eff3519c202c508b6e4602d5c1f1e17cfc18cb7a6953d77a548f2"} Jul 08 11:30:29.055049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:29.054989 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" podStartSLOduration=1.487520096 podStartE2EDuration="2.054970346s" totalImagesPullingTime="567.45025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:30:29.054034706 +0000 UTC m=+1045.435681477" watchObservedRunningTime="2026-07-08 11:30:29.054970346 +0000 UTC m=+1045.436617117" Jul 08 11:30:40.047255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:40.047219 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b9657c48d-mz6lj" Jul 08 11:30:59.921240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:30:59.921137 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:00.015293 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:00.015254 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:02.980344 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:02.980300 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:03.036333 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:03.036295 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:11.419583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:11.419545 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:11.523620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:11.523582 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:11.636571 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:11.636529 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:13.629793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:13.629754 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:13.735534 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:13.735491 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:17.594476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:17.594418 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:17.788147 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:17.788101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:17.932848 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:17.932766 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:23.716067 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:23.716028 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:31:23.823920 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:31:23.823887 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:32:21.950298 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:32:21.950256 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:33:04.149030 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:04.149000 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:33:04.151218 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:04.151192 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:33:15.410084 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:15.409997 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:33:15.464332 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:15.464285 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:33:49.808995 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:49.808955 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:33:49.910515 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:33:49.910475 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:34:49.960761 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:34:49.960726 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:35:15.926250 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:15.926209 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:35:15.926727 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:15.926426 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerName="manager" containerID="cri-o://7abb7d4205c66233fbad9d6a65018da7246895af1e194a059e4d79c940e42081" gracePeriod=10 Jul 08 11:35:18.288250 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.288201 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.39:8081/readyz\": dial tcp 10.132.0.39:8081: connect: connection refused" Jul 08 11:35:18.649410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.649323 2559 generic.go:356] "Generic (PLEG): container finished" podID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerID="7abb7d4205c66233fbad9d6a65018da7246895af1e194a059e4d79c940e42081" exitCode=0 Jul 08 11:35:18.649410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.649396 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" event={"ID":"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c","Type":"ContainerDied","Data":"7abb7d4205c66233fbad9d6a65018da7246895af1e194a059e4d79c940e42081"} Jul 08 11:35:18.890410 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.890384 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:35:18.927728 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.927616 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6w87\" (UniqueName: \"kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87\") pod \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " Jul 08 11:35:18.927889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.927768 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume\") pod \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\" (UID: \"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c\") " Jul 08 11:35:18.928136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.928106 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" (UID: "278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:35:18.929942 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:18.929917 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87" (OuterVolumeSpecName: "kube-api-access-n6w87") pod "278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" (UID: "278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c"). InnerVolumeSpecName "kube-api-access-n6w87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:35:19.029256 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.029222 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-extensions-socket-volume\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:35:19.029256 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.029252 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6w87\" (UniqueName: \"kubernetes.io/projected/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c-kube-api-access-n6w87\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:35:19.655762 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.655729 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" Jul 08 11:35:19.656262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.655764 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m" event={"ID":"278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c","Type":"ContainerDied","Data":"25262b37d5e78fd961331a8dc9afaa4f9389755dc9fa5de14a4c47bd16534990"} Jul 08 11:35:19.656262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.655838 2559 scope.go:117] "RemoveContainer" containerID="7abb7d4205c66233fbad9d6a65018da7246895af1e194a059e4d79c940e42081" Jul 08 11:35:19.677612 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.677587 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:35:19.681426 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:19.681405 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lt27m"] Jul 08 11:35:20.170303 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:35:20.170269 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" path="/var/lib/kubelet/pods/278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c/volumes" Jul 08 11:36:17.077707 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.077674 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh"] Jul 08 11:36:17.078115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.078078 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerName="manager" Jul 08 11:36:17.078115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.078089 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerName="manager" Jul 08 11:36:17.078193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.078150 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="278e12d0-7cf0-4bab-9cd8-ac0ff4fb287c" containerName="manager" Jul 08 11:36:17.082331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.082314 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.085281 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.085263 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-trpgp\"" Jul 08 11:36:17.097009 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.096988 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh"] Jul 08 11:36:17.113104 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.113071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.113241 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.113122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwxwg\" (UniqueName: \"kubernetes.io/projected/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-kube-api-access-jwxwg\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.213770 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.213732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.213983 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.213807 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwxwg\" (UniqueName: \"kubernetes.io/projected/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-kube-api-access-jwxwg\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.214163 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.214135 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.224265 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.224244 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwxwg\" (UniqueName: \"kubernetes.io/projected/af880455-2db5-4b1e-83c7-ffd3e24e2e5b-kube-api-access-jwxwg\") pod \"kuadrant-operator-controller-manager-86878c769f-pvrnh\" (UID: \"af880455-2db5-4b1e-83c7-ffd3e24e2e5b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.392282 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.392165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:17.546335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.546292 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh"] Jul 08 11:36:17.547006 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:36:17.546976 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf880455_2db5_4b1e_83c7_ffd3e24e2e5b.slice/crio-1a7cce6f15b597442dbbdac394ef2caebfec38fa60ffcd14c58abce7a01f9e45 WatchSource:0}: Error finding container 1a7cce6f15b597442dbbdac394ef2caebfec38fa60ffcd14c58abce7a01f9e45: Status 404 returned error can't find the container with id 1a7cce6f15b597442dbbdac394ef2caebfec38fa60ffcd14c58abce7a01f9e45 Jul 08 11:36:17.974927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.974896 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" event={"ID":"af880455-2db5-4b1e-83c7-ffd3e24e2e5b","Type":"ContainerStarted","Data":"3fe09a7c226bcd7ba0fd7ad57d94b45904b0e98a136efadc4499bf7eab1207b1"} Jul 08 11:36:17.975103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.974950 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" event={"ID":"af880455-2db5-4b1e-83c7-ffd3e24e2e5b","Type":"ContainerStarted","Data":"1a7cce6f15b597442dbbdac394ef2caebfec38fa60ffcd14c58abce7a01f9e45"} Jul 08 11:36:17.975103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:17.974982 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:36:18.001892 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:18.001838 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" podStartSLOduration=1.001819869 podStartE2EDuration="1.001819869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:36:17.998355466 +0000 UTC m=+1394.380002236" watchObservedRunningTime="2026-07-08 11:36:18.001819869 +0000 UTC m=+1394.383466705" Jul 08 11:36:28.983697 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:36:28.983665 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pvrnh" Jul 08 11:37:27.271136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:37:27.271104 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:37:59.130009 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:37:59.129974 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:38:04.185085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:38:04.185049 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:38:04.186238 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:38:04.186220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:38:51.256025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:38:51.255950 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:38:51.357371 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:38:51.357338 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:06.185355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:06.185322 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:06.267975 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:06.267937 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:06.621001 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:06.620971 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:12.306863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.306825 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:12.407622 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.407584 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:12.482470 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.482439 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:12.486721 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.486700 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.489136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.489120 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-aab7f969-openshift-default-dockercfg-rqcxb\"" Jul 08 11:42:12.513693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.513661 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:12.530494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.530459 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:12.562488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562521 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562551 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qrqd\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562599 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562648 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562892 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562722 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562892 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562745 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.562892 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.562762 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.628031 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.627988 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:12.664128 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664090 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664128 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qrqd\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664554 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664628 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664614 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664669 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.664858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.664698 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.665168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.665149 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.666782 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.666760 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.666982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.666965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.679013 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.678989 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.679130 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.679064 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qrqd\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd\") pod \"e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.797108 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.797068 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:12.947531 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.947504 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:12.948566 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:42:12.948539 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod711e6feb_d019_4b22_8885_04c36a33c03c.slice/crio-ebca297f245fb764b749932b54175f23293d4d8775ed4010354e2d40db973a50 WatchSource:0}: Error finding container ebca297f245fb764b749932b54175f23293d4d8775ed4010354e2d40db973a50: Status 404 returned error can't find the container with id ebca297f245fb764b749932b54175f23293d4d8775ed4010354e2d40db973a50 Jul 08 11:42:12.950559 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.950528 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:12.950670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.950588 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:12.950670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:12.950621 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:13.618402 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.618368 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:13.704964 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.704926 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:13.781145 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.781089 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:13.951034 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.950947 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" event={"ID":"711e6feb-d019-4b22-8885-04c36a33c03c","Type":"ContainerStarted","Data":"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1"} Jul 08 11:42:13.951184 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.951040 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" event={"ID":"711e6feb-d019-4b22-8885-04c36a33c03c","Type":"ContainerStarted","Data":"ebca297f245fb764b749932b54175f23293d4d8775ed4010354e2d40db973a50"} Jul 08 11:42:13.975441 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:13.975388 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" podStartSLOduration=1.975373221 podStartE2EDuration="1.975373221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:42:13.971794396 +0000 UTC m=+1750.353441171" watchObservedRunningTime="2026-07-08 11:42:13.975373221 +0000 UTC m=+1750.357019992" Jul 08 11:42:14.113966 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.113921 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:14.277126 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.277088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:14.377917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.377877 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:14.798205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.798171 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:14.803656 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.803610 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:14.956468 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.956442 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:14.957377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:14.957355 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:16.370355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:16.370299 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:18.675431 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:18.675392 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:18.839220 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:18.839177 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:18.839455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:18.839423 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" podUID="711e6feb-d019-4b22-8885-04c36a33c03c" containerName="istio-proxy" containerID="cri-o://3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1" gracePeriod=30 Jul 08 11:42:18.860826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:18.860775 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:18.963385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:18.963347 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:19.103333 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:19.103298 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:21.313687 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:21.313652 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:25.026865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.026829 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:25.141521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.141479 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:25.203867 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.203836 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:25.209950 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.209926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.212741 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.212711 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e3b7697a-openshift-default-dockercfg-hcgpr\"" Jul 08 11:42:25.224384 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.224359 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:25.247126 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.247096 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:25.298150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298130 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298369 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298209 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298369 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298260 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298369 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298344 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298562 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.298668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.298604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwpmh\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.344941 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.344901 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:25.399759 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399795 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399902 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwpmh\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.399963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400085 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400250 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400506 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.400608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.402573 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.402826 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.402719 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.409059 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.409035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwpmh\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.409170 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.409040 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token\") pod \"e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.522219 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.522182 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:25.729009 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.728977 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:25.885261 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.885234 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:25.886602 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:42:25.886576 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f33a04_af6a_46af_8304_6d7cf6c88f32.slice/crio-029dc2116b51db82a0e3b105634afd4c367085c40829bae6ecce01384ba4e91e WatchSource:0}: Error finding container 029dc2116b51db82a0e3b105634afd4c367085c40829bae6ecce01384ba4e91e: Status 404 returned error can't find the container with id 029dc2116b51db82a0e3b105634afd4c367085c40829bae6ecce01384ba4e91e Jul 08 11:42:25.888833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.888802 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:25.888900 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.888871 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:25.888934 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.888903 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:25.906663 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906621 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.906777 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906675 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.906777 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906732 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.906896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906791 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qrqd\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.906896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906831 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.907035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906893 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.907035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906938 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.907035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.906964 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.907035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.907011 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert\") pod \"711e6feb-d019-4b22-8885-04c36a33c03c\" (UID: \"711e6feb-d019-4b22-8885-04c36a33c03c\") " Jul 08 11:42:25.909096 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909069 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:42:25.909243 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909217 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:25.909319 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909292 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:25.909378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909345 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data" (OuterVolumeSpecName: "istio-data") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:25.909441 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909388 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd" (OuterVolumeSpecName: "kube-api-access-2qrqd") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "kube-api-access-2qrqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:25.909441 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909431 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:25.909794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.909766 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:42:25.910930 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.910905 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token" (OuterVolumeSpecName: "istio-token") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:25.911067 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:25.911049 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "711e6feb-d019-4b22-8885-04c36a33c03c" (UID: "711e6feb-d019-4b22-8885-04c36a33c03c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:26.008274 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008239 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/711e6feb-d019-4b22-8885-04c36a33c03c-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008274 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008272 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008295 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008308 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qrqd\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-kube-api-access-2qrqd\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008321 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/711e6feb-d019-4b22-8885-04c36a33c03c-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008332 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008342 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008352 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/711e6feb-d019-4b22-8885-04c36a33c03c-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.008498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.008364 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/711e6feb-d019-4b22-8885-04c36a33c03c-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:26.021454 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.021424 2559 generic.go:356] "Generic (PLEG): container finished" podID="711e6feb-d019-4b22-8885-04c36a33c03c" containerID="3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1" exitCode=0 Jul 08 11:42:26.021703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.021495 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" event={"ID":"711e6feb-d019-4b22-8885-04c36a33c03c","Type":"ContainerDied","Data":"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1"} Jul 08 11:42:26.021703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.021548 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" Jul 08 11:42:26.021703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.021573 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q" event={"ID":"711e6feb-d019-4b22-8885-04c36a33c03c","Type":"ContainerDied","Data":"ebca297f245fb764b749932b54175f23293d4d8775ed4010354e2d40db973a50"} Jul 08 11:42:26.021703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.021604 2559 scope.go:117] "RemoveContainer" containerID="3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1" Jul 08 11:42:26.023930 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.023780 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" event={"ID":"07f33a04-af6a-46af-8304-6d7cf6c88f32","Type":"ContainerStarted","Data":"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056"} Jul 08 11:42:26.023930 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.023860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" event={"ID":"07f33a04-af6a-46af-8304-6d7cf6c88f32","Type":"ContainerStarted","Data":"029dc2116b51db82a0e3b105634afd4c367085c40829bae6ecce01384ba4e91e"} Jul 08 11:42:26.033365 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.033347 2559 scope.go:117] "RemoveContainer" containerID="3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1" Jul 08 11:42:26.033786 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:26.033713 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1\": container with ID starting with 3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1 not found: ID does not exist" containerID="3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1" Jul 08 11:42:26.033786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.033768 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1"} err="failed to get container status \"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1\": rpc error: code = NotFound desc = could not find container \"3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1\": container with ID starting with 3c40462a903d5a0dec164747649674232e2ebd32e6809c19ac647f4beda38dc1 not found: ID does not exist" Jul 08 11:42:26.050222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.050173 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" podStartSLOduration=1.050158704 podStartE2EDuration="1.050158704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:42:26.047062936 +0000 UTC m=+1762.428709705" watchObservedRunningTime="2026-07-08 11:42:26.050158704 +0000 UTC m=+1762.431805474" Jul 08 11:42:26.063858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.063808 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:26.067884 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.067858 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-aab7f969-openshift-default-8d676ddd9-j2s4q"] Jul 08 11:42:26.171047 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.170971 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="711e6feb-d019-4b22-8885-04c36a33c03c" path="/var/lib/kubelet/pods/711e6feb-d019-4b22-8885-04c36a33c03c/volumes" Jul 08 11:42:26.440557 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.440457 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:26.511593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.511552 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:26.512221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.512197 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="711e6feb-d019-4b22-8885-04c36a33c03c" containerName="istio-proxy" Jul 08 11:42:26.512324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.512225 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="711e6feb-d019-4b22-8885-04c36a33c03c" containerName="istio-proxy" Jul 08 11:42:26.512993 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.512379 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="711e6feb-d019-4b22-8885-04c36a33c03c" containerName="istio-proxy" Jul 08 11:42:26.517381 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.517362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.520534 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.520512 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e3b7697a-serving-cert\"" Jul 08 11:42:26.522509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.522485 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:26.523795 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.523769 2559 patch_prober.go:28] interesting pod/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 08 11:42:26.523898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.523815 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 08 11:42:26.525961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.525850 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:26.536865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.536602 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:26.613359 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.613323 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:26.613562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.613486 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.613562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.613524 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.613713 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.613573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwsb\" (UniqueName: \"kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.714563 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.714529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.714802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.714568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.714802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.714683 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwsb\" (UniqueName: \"kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.714802 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:26.714720 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-serving-cert: secret "maas-api-e2e-ait-e3b7697a-serving-cert" not found Jul 08 11:42:26.714802 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:26.714787 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls podName:1d42ee67-cc5b-46f0-85f8-68f85cf35eb8 nodeName:}" failed. No retries permitted until 2026-07-08 11:42:27.214767347 +0000 UTC m=+1763.596414119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls") pod "maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" (UID: "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8") : secret "maas-api-e2e-ait-e3b7697a-serving-cert" not found Jul 08 11:42:26.715306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.715282 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:26.723683 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:26.723627 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwsb\" (UniqueName: \"kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:27.013769 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.013674 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:27.129446 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.129390 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:27.220012 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.219968 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:27.222828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.222728 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") pod \"maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:27.262875 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.262840 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:27.433230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.433144 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:27.526563 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.526527 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:27.602781 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.602755 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:27.604033 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:42:27.604010 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d42ee67_cc5b_46f0_85f8_68f85cf35eb8.slice/crio-92fd4e7ca9f60a4bdd5f0ab6ee9e3c4db6d7f40e6cad74609ef0796bc58e268a WatchSource:0}: Error finding container 92fd4e7ca9f60a4bdd5f0ab6ee9e3c4db6d7f40e6cad74609ef0796bc58e268a: Status 404 returned error can't find the container with id 92fd4e7ca9f60a4bdd5f0ab6ee9e3c4db6d7f40e6cad74609ef0796bc58e268a Jul 08 11:42:27.605541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:27.605524 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 11:42:28.050303 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:28.050241 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" event={"ID":"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8","Type":"ContainerStarted","Data":"92fd4e7ca9f60a4bdd5f0ab6ee9e3c4db6d7f40e6cad74609ef0796bc58e268a"} Jul 08 11:42:28.050786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:28.050756 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:28.051850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:28.051829 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:28.186425 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:28.186397 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:29.058345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:29.058310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" event={"ID":"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8","Type":"ContainerStarted","Data":"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248"} Jul 08 11:42:29.058552 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:29.058534 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:29.084810 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:29.084758 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" podStartSLOduration=2.677800171 podStartE2EDuration="3.084740646s" totalImagesPullingTime="406.940475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:42:29.082513478 +0000 UTC m=+1765.464160240" watchObservedRunningTime="2026-07-08 11:42:29.084740646 +0000 UTC m=+1765.466387431" Jul 08 11:42:31.433184 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:31.433145 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:31.965663 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:31.965182 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:31.965663 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:31.965406 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" podUID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" containerName="maas-api" containerID="cri-o://13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248" gracePeriod=30 Jul 08 11:42:32.287891 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.287862 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:32.370788 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.370753 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzwsb\" (UniqueName: \"kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb\") pod \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " Jul 08 11:42:32.370959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.370806 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") pod \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " Jul 08 11:42:32.370959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.370825 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca\") pod \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\" (UID: \"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8\") " Jul 08 11:42:32.371219 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.371191 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" (UID: "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:42:32.373083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.373061 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb" (OuterVolumeSpecName: "kube-api-access-rzwsb") pod "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" (UID: "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8"). InnerVolumeSpecName "kube-api-access-rzwsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:32.373188 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.373168 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" (UID: "1d42ee67-cc5b-46f0-85f8-68f85cf35eb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:42:32.471488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.471448 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzwsb\" (UniqueName: \"kubernetes.io/projected/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-kube-api-access-rzwsb\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:32.471488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.471480 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-maas-api-tls\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:32.471488 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.471493 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8-openshift-service-ca\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:32.609099 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:32.608994 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:33.087403 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.087363 2559 generic.go:356] "Generic (PLEG): container finished" podID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" containerID="13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248" exitCode=0 Jul 08 11:42:33.087586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.087399 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" event={"ID":"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8","Type":"ContainerDied","Data":"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248"} Jul 08 11:42:33.087586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.087459 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" event={"ID":"1d42ee67-cc5b-46f0-85f8-68f85cf35eb8","Type":"ContainerDied","Data":"92fd4e7ca9f60a4bdd5f0ab6ee9e3c4db6d7f40e6cad74609ef0796bc58e268a"} Jul 08 11:42:33.087586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.087481 2559 scope.go:117] "RemoveContainer" containerID="13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248" Jul 08 11:42:33.087586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.087411 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j" Jul 08 11:42:33.097320 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.097302 2559 scope.go:117] "RemoveContainer" containerID="13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248" Jul 08 11:42:33.097621 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:33.097602 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248\": container with ID starting with 13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248 not found: ID does not exist" containerID="13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248" Jul 08 11:42:33.097729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.097628 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248"} err="failed to get container status \"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248\": rpc error: code = NotFound desc = could not find container \"13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248\": container with ID starting with 13bb12254afd9ebf44a3d9c3b6f818408fcbee650541b58f03863cc340091248 not found: ID does not exist" Jul 08 11:42:33.109862 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.109826 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:33.114151 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:33.114120 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3b7697a-7d5b4b679b-rcz2j"] Jul 08 11:42:34.170559 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:34.170526 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" path="/var/lib/kubelet/pods/1d42ee67-cc5b-46f0-85f8-68f85cf35eb8/volumes" Jul 08 11:42:37.572482 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:37.572440 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:37.573004 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:37.572706 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerName="istio-proxy" containerID="cri-o://c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056" gracePeriod=30 Jul 08 11:42:37.604393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:37.604356 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:37.710738 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:37.710694 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:38.840834 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.840811 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:38.931887 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.931858 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.931893 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.931947 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwpmh\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.931969 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.931989 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932017 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932048 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932102 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932170 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token\") pod \"07f33a04-af6a-46af-8304-6d7cf6c88f32\" (UID: \"07f33a04-af6a-46af-8304-6d7cf6c88f32\") " Jul 08 11:42:38.932312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932196 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:38.932537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932399 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:42:38.932537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932447 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:38.932537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932493 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:38.932537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932511 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f33a04-af6a-46af-8304-6d7cf6c88f32-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:38.932804 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.932591 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:38.933284 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.933249 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data" (OuterVolumeSpecName: "istio-data") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:38.934445 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.934421 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh" (OuterVolumeSpecName: "kube-api-access-xwpmh") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "kube-api-access-xwpmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:38.934574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.934467 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token" (OuterVolumeSpecName: "istio-token") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:38.934859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.934837 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:42:38.935163 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:38.935145 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07f33a04-af6a-46af-8304-6d7cf6c88f32" (UID: "07f33a04-af6a-46af-8304-6d7cf6c88f32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:39.033491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033461 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033491 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033505 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwpmh\" (UniqueName: \"kubernetes.io/projected/07f33a04-af6a-46af-8304-6d7cf6c88f32-kube-api-access-xwpmh\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033521 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033533 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033543 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f33a04-af6a-46af-8304-6d7cf6c88f32-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.033745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.033553 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f33a04-af6a-46af-8304-6d7cf6c88f32-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:39.126301 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.126268 2559 generic.go:356] "Generic (PLEG): container finished" podID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerID="c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056" exitCode=0 Jul 08 11:42:39.126469 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.126312 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" Jul 08 11:42:39.126469 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.126320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" event={"ID":"07f33a04-af6a-46af-8304-6d7cf6c88f32","Type":"ContainerDied","Data":"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056"} Jul 08 11:42:39.126469 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.126364 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2" event={"ID":"07f33a04-af6a-46af-8304-6d7cf6c88f32","Type":"ContainerDied","Data":"029dc2116b51db82a0e3b105634afd4c367085c40829bae6ecce01384ba4e91e"} Jul 08 11:42:39.126469 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.126386 2559 scope.go:117] "RemoveContainer" containerID="c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056" Jul 08 11:42:39.136554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.136527 2559 scope.go:117] "RemoveContainer" containerID="c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056" Jul 08 11:42:39.136875 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:39.136857 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056\": container with ID starting with c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056 not found: ID does not exist" containerID="c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056" Jul 08 11:42:39.136945 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.136884 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056"} err="failed to get container status \"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056\": rpc error: code = NotFound desc = could not find container \"c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056\": container with ID starting with c82d67af02ce65df6012e4f2f7d4d24ac9f92a1246d4b27a18710339003c5056 not found: ID does not exist" Jul 08 11:42:39.148872 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.148811 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:39.153163 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.153143 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e3b7697a-openshift-default-c9455464c-hpvn2"] Jul 08 11:42:39.214414 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:39.214364 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:40.169067 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:40.169036 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" path="/var/lib/kubelet/pods/07f33a04-af6a-46af-8304-6d7cf6c88f32/volumes" Jul 08 11:42:43.576030 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.575996 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:43.639212 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.639162 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:43.764939 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.764904 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:43.765512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765493 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerName="istio-proxy" Jul 08 11:42:43.765555 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765518 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerName="istio-proxy" Jul 08 11:42:43.765555 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765537 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" containerName="maas-api" Jul 08 11:42:43.765555 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765546 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" containerName="maas-api" Jul 08 11:42:43.765720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765693 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f33a04-af6a-46af-8304-6d7cf6c88f32" containerName="istio-proxy" Jul 08 11:42:43.765720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.765708 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d42ee67-cc5b-46f0-85f8-68f85cf35eb8" containerName="maas-api" Jul 08 11:42:43.770586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.770566 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.773820 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.773802 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-411285cd-openshift-default-dockercfg-2pssh\"" Jul 08 11:42:43.783136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.783113 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:43.804049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.804003 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:43.887167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887075 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887167 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm58k\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887227 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887420 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.887612 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.887444 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.904019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.903991 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:43.988903 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.988863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.988912 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.988959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.988990 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989036 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989105 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989090 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm58k\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989376 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989434 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989459 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989514 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989846 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989824 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.989919 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.989831 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.991501 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.991476 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.991912 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.991893 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.998136 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.998115 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:43.998219 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:43.998202 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm58k\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k\") pod \"e2e-derive-411285cd-openshift-default-5756595b67-jbb26\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:44.082294 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.082260 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:44.251781 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.251750 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:44.252912 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:42:44.252883 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f19cd1_3658_44ad_9f83_df6e44bd481e.slice/crio-6417bce29fe14ac4f37afb6fb9888fc17a1c1d04c9199326c36485ea8f12c008 WatchSource:0}: Error finding container 6417bce29fe14ac4f37afb6fb9888fc17a1c1d04c9199326c36485ea8f12c008: Status 404 returned error can't find the container with id 6417bce29fe14ac4f37afb6fb9888fc17a1c1d04c9199326c36485ea8f12c008 Jul 08 11:42:44.255184 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.255151 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:44.255277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.255218 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:44.255277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.255246 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:42:44.892609 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.892577 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:44.971893 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:44.971852 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:45.036513 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.036484 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:45.138821 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.138779 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:45.164567 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.164487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" event={"ID":"97f19cd1-3658-44ad-9f83-df6e44bd481e","Type":"ContainerStarted","Data":"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8"} Jul 08 11:42:45.164567 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.164532 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" event={"ID":"97f19cd1-3658-44ad-9f83-df6e44bd481e","Type":"ContainerStarted","Data":"6417bce29fe14ac4f37afb6fb9888fc17a1c1d04c9199326c36485ea8f12c008"} Jul 08 11:42:45.186291 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.186242 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" podStartSLOduration=2.186226543 podStartE2EDuration="2.186226543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:42:45.184056108 +0000 UTC m=+1781.565702903" watchObservedRunningTime="2026-07-08 11:42:45.186226543 +0000 UTC m=+1781.567873315" Jul 08 11:42:45.458434 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.458405 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:45.556131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:45.556094 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:46.082553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:46.082512 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:46.087749 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:46.087719 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:46.170542 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:46.170503 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:46.171408 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:46.171388 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:46.677205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:46.677174 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:49.418565 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.418528 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:49.655517 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.655467 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:49.655802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.655769 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" podUID="97f19cd1-3658-44ad-9f83-df6e44bd481e" containerName="istio-proxy" containerID="cri-o://feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8" gracePeriod=30 Jul 08 11:42:49.668828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.668727 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:49.779968 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.779929 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:49.979127 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:49.979084 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:50.952978 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:50.952929 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:55.558472 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.558370 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:55.613618 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.613592 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:55.631042 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.630995 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:55.715512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715470 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715520 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715539 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715584 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715613 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm58k\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715687 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715737 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715784 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715826 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy\") pod \"97f19cd1-3658-44ad-9f83-df6e44bd481e\" (UID: \"97f19cd1-3658-44ad-9f83-df6e44bd481e\") " Jul 08 11:42:55.715967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715837 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:55.715967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.715956 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:55.716227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716079 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:55.716227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716184 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:42:55.716227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716192 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.716227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716211 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.716227 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716225 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.716506 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.716419 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data" (OuterVolumeSpecName: "istio-data") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:55.718154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.718131 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:42:55.718154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.718138 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:42:55.718324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.718150 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token" (OuterVolumeSpecName: "istio-token") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:55.718324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.718279 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k" (OuterVolumeSpecName: "kube-api-access-qm58k") pod "97f19cd1-3658-44ad-9f83-df6e44bd481e" (UID: "97f19cd1-3658-44ad-9f83-df6e44bd481e"). InnerVolumeSpecName "kube-api-access-qm58k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:42:55.817160 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817120 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm58k\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-kube-api-access-qm58k\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.817160 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817157 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.817160 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817170 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97f19cd1-3658-44ad-9f83-df6e44bd481e-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.817455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817181 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.817455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817191 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:55.817455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:55.817203 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97f19cd1-3658-44ad-9f83-df6e44bd481e-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:42:56.124266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.124165 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:56.219537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.219502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:56.222594 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.222564 2559 generic.go:356] "Generic (PLEG): container finished" podID="97f19cd1-3658-44ad-9f83-df6e44bd481e" containerID="feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8" exitCode=0 Jul 08 11:42:56.222748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.222612 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" Jul 08 11:42:56.222748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.222672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" event={"ID":"97f19cd1-3658-44ad-9f83-df6e44bd481e","Type":"ContainerDied","Data":"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8"} Jul 08 11:42:56.222748 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.222729 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26" event={"ID":"97f19cd1-3658-44ad-9f83-df6e44bd481e","Type":"ContainerDied","Data":"6417bce29fe14ac4f37afb6fb9888fc17a1c1d04c9199326c36485ea8f12c008"} Jul 08 11:42:56.222855 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.222756 2559 scope.go:117] "RemoveContainer" containerID="feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8" Jul 08 11:42:56.232572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.232545 2559 scope.go:117] "RemoveContainer" containerID="feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8" Jul 08 11:42:56.232848 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:42:56.232829 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8\": container with ID starting with feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8 not found: ID does not exist" containerID="feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8" Jul 08 11:42:56.232904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.232863 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8"} err="failed to get container status \"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8\": rpc error: code = NotFound desc = could not find container \"feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8\": container with ID starting with feadcfe2789f58ba5c1c905ccece7578a2e2bf9a351001d9ec3cb4e9aa6d36c8 not found: ID does not exist" Jul 08 11:42:56.241866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.241836 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:56.246150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.246124 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-411285cd-openshift-default-5756595b67-jbb26"] Jul 08 11:42:56.283784 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:56.283755 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:57.770925 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:57.770895 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:42:58.176208 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:42:58.176125 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97f19cd1-3658-44ad-9f83-df6e44bd481e" path="/var/lib/kubelet/pods/97f19cd1-3658-44ad-9f83-df6e44bd481e/volumes" Jul 08 11:43:03.526499 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:03.526470 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:03.656058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:03.656015 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:03.752343 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:03.752299 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:03.876111 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:03.876016 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:04.229700 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:04.229675 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:43:04.231474 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:04.231452 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:43:04.286523 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:04.286488 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:30.226189 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:30.226150 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:36.390234 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:36.390192 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:36.492499 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:36.492468 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:36.593697 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:36.593656 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:52.473470 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:52.473436 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:58.438368 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:58.438331 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:58.547316 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:58.547277 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:43:58.645676 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:43:58.645628 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:09.173694 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:09.173651 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:09.942322 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:09.942280 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:15.839868 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:15.839837 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:15.913065 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:15.913024 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:16.041610 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:16.041578 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:16.194179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:16.194077 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:16.971768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:16.971737 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:17.072027 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:17.071987 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:17.174103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:17.174053 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:18.786539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:18.786502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:23.851168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:23.851124 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:24.936742 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:24.936552 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:25.040229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:25.040179 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:25.679230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:25.679189 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:31.648580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:31.648544 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:31.751980 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:31.751946 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:31.955857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:31.955822 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:32.235755 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.235668 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:44:32.236736 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.236704 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97f19cd1-3658-44ad-9f83-df6e44bd481e" containerName="istio-proxy" Jul 08 11:44:32.236736 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.236732 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="97f19cd1-3658-44ad-9f83-df6e44bd481e" containerName="istio-proxy" Jul 08 11:44:32.236960 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.236871 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="97f19cd1-3658-44ad-9f83-df6e44bd481e" containerName="istio-proxy" Jul 08 11:44:32.240424 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.240404 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.243234 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.243213 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dc78c413-openshift-default-dockercfg-8wflf\"" Jul 08 11:44:32.263604 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.263576 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:44:32.267161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.267122 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:32.380912 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.380869 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.380930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.380956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.380990 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.381064 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.381100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5kjd\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.381127 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.381229 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.381330 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.381258 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.435204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.435164 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:32.482562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.482562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.482856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.482856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482744 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.482856 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.482969 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483007 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5kjd\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483022 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483181 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483140 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483341 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483234 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483341 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483307 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.483612 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.483591 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.485554 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.485528 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.485687 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.485628 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.491352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.491295 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.491716 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.491699 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5kjd\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd\") pod \"e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.551774 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.551739 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:32.702086 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.702059 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:44:32.703269 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:44:32.703245 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb41d94b_89ad_4ab3_8c8b_d6d1cf9ed62b.slice/crio-6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63 WatchSource:0}: Error finding container 6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63: Status 404 returned error can't find the container with id 6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63 Jul 08 11:44:32.705777 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.705742 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:32.705866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.705815 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:32.705866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.705844 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:32.754355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:32.754322 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerStarted","Data":"6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63"} Jul 08 11:44:33.760912 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:33.760874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerStarted","Data":"21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047"} Jul 08 11:44:33.787699 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:33.787646 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" podStartSLOduration=1.787613949 podStartE2EDuration="1.787613949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:44:33.783892663 +0000 UTC m=+1890.165539459" watchObservedRunningTime="2026-07-08 11:44:33.787613949 +0000 UTC m=+1890.169260720" Jul 08 11:44:34.552889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:34.552850 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:34.558140 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:34.558117 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:34.766668 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:34.766615 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:34.767580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:34.767563 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:35.913947 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:35.913909 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:36.085971 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:36.085931 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:36.507904 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:36.507860 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:36.663732 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:36.663694 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:36.762961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:36.762853 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:40.322672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:40.322617 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:41.014939 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:41.014908 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:41.127918 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:41.127886 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:41.453432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:41.453339 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:47.168596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:47.168557 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:47.268265 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:47.268231 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:47.567703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:47.567664 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:47.956507 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:47.956475 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:48.056125 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:48.056090 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:49.467335 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:49.467295 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:49.570937 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:49.570896 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:50.782027 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.780919 2559 patch_prober.go:28] interesting pod/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:44:50.782027 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.780985 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:44:50.863684 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.863600 2559 generic.go:356] "Generic (PLEG): container finished" podID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerID="21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047" exitCode=0 Jul 08 11:44:50.863849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.863679 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerDied","Data":"21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047"} Jul 08 11:44:50.864071 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.864057 2559 scope.go:117] "RemoveContainer" containerID="21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047" Jul 08 11:44:50.865075 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.865047 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:50.865148 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.865130 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:50.865210 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:50.865193 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:44:51.871555 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:51.871523 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerStarted","Data":"b1c821a075ff303e1728c26863f2fbb6be6b2a0ac9469cb1c63a93e8b9d54a8b"} Jul 08 11:44:52.553130 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:52.553093 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:44:52.553583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:52.553497 2559 patch_prober.go:28] interesting pod/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 11:44:52.553789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:52.553766 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 11:44:53.553507 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:53.553034 2559 patch_prober.go:28] interesting pod/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 08 11:44:53.553507 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:53.553102 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 08 11:44:53.812146 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:53.812039 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:53.911847 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:53.911811 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:54.060258 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.060211 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:44:54.061040 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.061009 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" containerID="cri-o://b1c821a075ff303e1728c26863f2fbb6be6b2a0ac9469cb1c63a93e8b9d54a8b" gracePeriod=30 Jul 08 11:44:54.077887 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.077854 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:54.453491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.453392 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:54.582623 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.582575 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:44:54.682052 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:44:54.682019 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.267141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.267100 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.338886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.338852 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.428048 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.428005 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.526770 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.526671 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.623738 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.623704 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:00.922016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.921989 2559 generic.go:356] "Generic (PLEG): container finished" podID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerID="b1c821a075ff303e1728c26863f2fbb6be6b2a0ac9469cb1c63a93e8b9d54a8b" exitCode=0 Jul 08 11:45:00.922173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.922065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerDied","Data":"b1c821a075ff303e1728c26863f2fbb6be6b2a0ac9469cb1c63a93e8b9d54a8b"} Jul 08 11:45:00.922173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.922126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" event={"ID":"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b","Type":"ContainerDied","Data":"6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63"} Jul 08 11:45:00.922173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.922147 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f01aef8f2ef5f191818f8721e2f78cf4baf0ce3d018a7915f293cab5d204a63" Jul 08 11:45:00.922173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.922161 2559 scope.go:117] "RemoveContainer" containerID="21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047" Jul 08 11:45:00.934369 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:45:00.934340 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047\": container with ID starting with 21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047 not found: ID does not exist" containerID="21338c77a0f506bd71075152a4ede02a2aebc9d7c3c9eb42ed60f509fcddf047" Jul 08 11:45:00.937898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:00.937873 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:45:01.046356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046253 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046309 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046339 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5kjd\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046360 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046381 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046400 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046457 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046505 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046559 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs\") pod \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\" (UID: \"db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b\") " Jul 08 11:45:01.046729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046696 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:45:01.047008 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046902 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:45:01.047008 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.046942 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.047103 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.047080 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:45:01.047683 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.047622 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:45:01.047928 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.047899 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data" (OuterVolumeSpecName: "istio-data") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:45:01.048962 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.048941 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd" (OuterVolumeSpecName: "kube-api-access-l5kjd") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "kube-api-access-l5kjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:45:01.049062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.048979 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:45:01.049333 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.049295 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:45:01.049435 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.049376 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token" (OuterVolumeSpecName: "istio-token") pod "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" (UID: "db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:45:01.147849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147809 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5kjd\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-kube-api-access-l5kjd\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.147849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147842 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.147849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147851 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.147849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147860 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.148120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147868 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.148120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147876 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.148120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147884 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.148120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.147893 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:45:01.929703 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.929674 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z" Jul 08 11:45:01.965229 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.965190 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:45:01.972831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:01.972804 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dc78c413-openshift-default-6478d95c75-xfd8z"] Jul 08 11:45:02.169301 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:02.169265 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" path="/var/lib/kubelet/pods/db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b/volumes" Jul 08 11:45:06.516057 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:06.516024 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:06.583880 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:06.583839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:13.301325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.301288 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:13.597115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597035 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:45:13.597431 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597419 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.597477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597433 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.597477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597446 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.597477 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597452 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.597570 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597517 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.597570 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.597526 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="db41d94b-89ad-4ab3-8c8b-d6d1cf9ed62b" containerName="istio-proxy" Jul 08 11:45:13.600646 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.600616 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.603109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.603094 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5cec6d9a-openshift-default-dockercfg-rmccv\"" Jul 08 11:45:13.612238 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.612213 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:45:13.621061 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.621031 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:13.721464 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.721411 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:13.768734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.768681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.768938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.768823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.768938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.768880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.768938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.768911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.768938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.768937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.769152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.769052 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.769152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.769092 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.769152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.769120 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.769261 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.769178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm4mt\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870238 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm4mt\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870351 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870899 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870693 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870899 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870723 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.870899 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870802 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.871058 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.870967 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.871101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.871061 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.871154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.871131 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.872893 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.872871 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.873050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.873034 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.878506 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.878485 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.878772 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.878751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm4mt\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt\") pod \"e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:13.912828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:13.912792 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:14.064736 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:14.064703 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:45:14.065597 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:45:14.065571 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e313269_b052_4ca0_8348_8167b5fe0059.slice/crio-b5afe32ccdb07aea040e3a2304297a650891a5a70ba5e5121e3edd5af0aaf4e4 WatchSource:0}: Error finding container b5afe32ccdb07aea040e3a2304297a650891a5a70ba5e5121e3edd5af0aaf4e4: Status 404 returned error can't find the container with id b5afe32ccdb07aea040e3a2304297a650891a5a70ba5e5121e3edd5af0aaf4e4 Jul 08 11:45:14.067785 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:14.067754 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:14.067866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:14.067827 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:14.067866 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:14.067857 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:15.009961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:15.009919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerStarted","Data":"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa"} Jul 08 11:45:15.010409 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:15.009983 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerStarted","Data":"b5afe32ccdb07aea040e3a2304297a650891a5a70ba5e5121e3edd5af0aaf4e4"} Jul 08 11:45:15.031332 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:15.031287 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" podStartSLOduration=2.031272969 podStartE2EDuration="2.031272969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:45:15.029425872 +0000 UTC m=+1931.411072643" watchObservedRunningTime="2026-07-08 11:45:15.031272969 +0000 UTC m=+1931.412919740" Jul 08 11:45:15.913816 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:15.913781 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:15.918694 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:15.918669 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:16.016272 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:16.016244 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:16.017088 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:16.017071 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:17.041814 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:17.041755 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:17.143673 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:17.143604 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:17.431246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:17.431165 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:17.657016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:17.656978 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:18.005265 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:18.005225 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:18.109142 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:18.109105 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:21.462163 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:21.462126 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:30.102331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.102296 2559 generic.go:356] "Generic (PLEG): container finished" podID="8e313269-b052-4ca0-8348-8167b5fe0059" containerID="221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa" exitCode=0 Jul 08 11:45:30.102784 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.102351 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerDied","Data":"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa"} Jul 08 11:45:30.102865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.102844 2559 scope.go:117] "RemoveContainer" containerID="221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa" Jul 08 11:45:30.103833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.103807 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:30.103895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.103872 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:30.103931 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:30.103907 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:45:31.017024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:31.016985 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:31.109894 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:31.109857 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerStarted","Data":"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f"} Jul 08 11:45:31.913845 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:31.913800 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:31.915070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:31.915039 2559 patch_prober.go:28] interesting pod/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 11:45:31.915212 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:31.915096 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 11:45:32.913383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:32.913341 2559 patch_prober.go:28] interesting pod/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 11:45:32.913905 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:32.913411 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 11:45:33.913997 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:33.913959 2559 patch_prober.go:28] interesting pod/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 08 11:45:33.914377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:33.914022 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 08 11:45:34.917917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:34.917888 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:34.918401 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:34.918283 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:34.918979 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:34.918962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:45:48.746454 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:48.746409 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:48.847226 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:48.847192 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:52.205454 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.205420 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:45:52.209115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.209095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.213205 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.213180 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5cec6d9a\"/\"e2e-lifecycle-model-5cec6d9a-kserve-self-signed-certs\"" Jul 08 11:45:52.213379 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.213180 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5cec6d9a\"/\"kube-root-ca.crt\"" Jul 08 11:45:52.213539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.213250 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5cec6d9a\"/\"openshift-service-ca.crt\"" Jul 08 11:45:52.213670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.213297 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5cec6d9a\"/\"default-dockercfg-4rg54\"" Jul 08 11:45:52.216130 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.216103 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:45:52.231615 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.231585 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:45:52.345442 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.345403 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.345720 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.345698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.345776 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.345739 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k4bn\" (UniqueName: \"kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.345942 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.345923 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.346021 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.346008 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.346057 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.346041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447475 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447510 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k4bn\" (UniqueName: \"kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447806 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447806 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447687 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447806 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.447806 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.447794 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.448083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.448062 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.448083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.448075 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.448197 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.448161 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.450006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.449984 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.450428 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.450406 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.455412 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.455388 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k4bn\" (UniqueName: \"kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn\") pod \"e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.520961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.520873 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:45:52.667593 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:52.667566 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:45:52.669504 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:45:52.669477 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70d3946b_cd74_4dba_bf16_d25a6b064b7a.slice/crio-b5fa7dab7fc0ef32201cabb7e2ef66f562d293b2a9e00c14ed934d7203834924 WatchSource:0}: Error finding container b5fa7dab7fc0ef32201cabb7e2ef66f562d293b2a9e00c14ed934d7203834924: Status 404 returned error can't find the container with id b5fa7dab7fc0ef32201cabb7e2ef66f562d293b2a9e00c14ed934d7203834924 Jul 08 11:45:53.230821 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:53.230777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" event={"ID":"70d3946b-cd74-4dba-bf16-d25a6b064b7a","Type":"ContainerStarted","Data":"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960"} Jul 08 11:45:53.231240 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:53.230834 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" event={"ID":"70d3946b-cd74-4dba-bf16-d25a6b064b7a","Type":"ContainerStarted","Data":"b5fa7dab7fc0ef32201cabb7e2ef66f562d293b2a9e00c14ed934d7203834924"} Jul 08 11:45:53.250955 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:45:53.250896 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" podStartSLOduration=1.250880441 podStartE2EDuration="1.250880441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:45:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:45:53.247724966 +0000 UTC m=+1969.629371739" watchObservedRunningTime="2026-07-08 11:45:53.250880441 +0000 UTC m=+1969.632527211" Jul 08 11:46:02.521717 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:02.521674 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:02.522246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:02.521735 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:02.534538 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:02.534495 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:03.302238 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:03.302202 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:19.444054 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:19.444014 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:20.024377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:20.024336 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:20.125910 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:20.125863 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:20.307081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:20.306993 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:25.625875 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:25.625839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:30.611879 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:30.611840 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:46:30.693217 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:30.693178 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:46:45.633046 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.633007 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" podUID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" containerName="main" containerID="cri-o://53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960" gracePeriod=45 Jul 08 11:46:45.886993 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.886930 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:45.934967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.934935 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935005 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935036 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k4bn\" (UniqueName: \"kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935082 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935172 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935204 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir\") pod \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\" (UID: \"70d3946b-cd74-4dba-bf16-d25a6b064b7a\") " Jul 08 11:46:45.935326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935238 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:46:45.935485 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935462 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home" (OuterVolumeSpecName: "home") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:46:45.935547 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935523 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-home\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:45.935547 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935542 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-model-cache\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:45.935689 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.935655 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:46:45.937476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.937451 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm" (OuterVolumeSpecName: "dshm") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:46:45.937609 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.937511 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 11:46:45.937714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:45.937695 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn" (OuterVolumeSpecName: "kube-api-access-4k4bn") pod "70d3946b-cd74-4dba-bf16-d25a6b064b7a" (UID: "70d3946b-cd74-4dba-bf16-d25a6b064b7a"). InnerVolumeSpecName "kube-api-access-4k4bn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:46:46.036331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.036295 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tmp-dir\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:46.036331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.036321 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d3946b-cd74-4dba-bf16-d25a6b064b7a-dshm\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:46.036528 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.036341 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k4bn\" (UniqueName: \"kubernetes.io/projected/70d3946b-cd74-4dba-bf16-d25a6b064b7a-kube-api-access-4k4bn\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:46.036528 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.036349 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d3946b-cd74-4dba-bf16-d25a6b064b7a-tls-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:46:46.535879 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.535848 2559 generic.go:356] "Generic (PLEG): container finished" podID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" containerID="53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960" exitCode=0 Jul 08 11:46:46.536085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.535888 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" Jul 08 11:46:46.536085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.535936 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" event={"ID":"70d3946b-cd74-4dba-bf16-d25a6b064b7a","Type":"ContainerDied","Data":"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960"} Jul 08 11:46:46.536085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.535977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b" event={"ID":"70d3946b-cd74-4dba-bf16-d25a6b064b7a","Type":"ContainerDied","Data":"b5fa7dab7fc0ef32201cabb7e2ef66f562d293b2a9e00c14ed934d7203834924"} Jul 08 11:46:46.536085 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.535993 2559 scope.go:117] "RemoveContainer" containerID="53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960" Jul 08 11:46:46.545688 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.545670 2559 scope.go:117] "RemoveContainer" containerID="53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960" Jul 08 11:46:46.545923 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:46:46.545902 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960\": container with ID starting with 53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960 not found: ID does not exist" containerID="53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960" Jul 08 11:46:46.545992 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.545934 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960"} err="failed to get container status \"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960\": rpc error: code = NotFound desc = could not find container \"53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960\": container with ID starting with 53dad7b267ecf27476320620deb6587383cc5afa321ed6cb998296c25ea85960 not found: ID does not exist" Jul 08 11:46:46.554700 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.554677 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:46:46.558373 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:46.558354 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5cec6d9a/e2e-lifecycle-model-5cec6d9a-kserve-65445f8bf9-g229b"] Jul 08 11:46:48.169235 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:46:48.169200 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" path="/var/lib/kubelet/pods/70d3946b-cd74-4dba-bf16-d25a6b064b7a/volumes" Jul 08 11:47:07.253680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:07.253627 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:08.174681 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:08.174644 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:13.804383 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:13.804310 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:13.864137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:13.864104 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:13.941296 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:13.941258 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:47:13.941562 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:13.941533 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" containerID="cri-o://aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f" gracePeriod=30 Jul 08 11:47:13.964306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:13.964266 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:15.228607 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:15.228539 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:15.321981 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:15.321930 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:15.437038 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:15.437002 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:16.147679 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:16.147612 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:16.247325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:16.247290 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:16.352684 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:16.352625 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:17.770387 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:17.770335 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:19.203998 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.203928 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:47:19.332324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332285 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332330 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332355 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332397 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332415 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332459 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332501 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm4mt\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332555 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332590 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy\") pod \"8e313269-b052-4ca0-8348-8167b5fe0059\" (UID: \"8e313269-b052-4ca0-8348-8167b5fe0059\") " Jul 08 11:47:19.332994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332752 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:47:19.332994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332786 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:47:19.332994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332833 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:47:19.332994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332905 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:47:19.332994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.332991 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.333413 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.333010 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e313269-b052-4ca0-8348-8167b5fe0059-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.333413 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.333024 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.333413 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.333038 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.334032 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.334002 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data" (OuterVolumeSpecName: "istio-data") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:47:19.335003 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.334930 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt" (OuterVolumeSpecName: "kube-api-access-wm4mt") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "kube-api-access-wm4mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:47:19.335003 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.334948 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:47:19.335003 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.334974 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token" (OuterVolumeSpecName: "istio-token") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:47:19.335223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.335028 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8e313269-b052-4ca0-8348-8167b5fe0059" (UID: "8e313269-b052-4ca0-8348-8167b5fe0059"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:47:19.434345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.434305 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm4mt\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-kube-api-access-wm4mt\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.434345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.434339 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e313269-b052-4ca0-8348-8167b5fe0059-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.434345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.434348 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.434345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.434356 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e313269-b052-4ca0-8348-8167b5fe0059-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.434624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.434364 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e313269-b052-4ca0-8348-8167b5fe0059-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:47:19.728960 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.728924 2559 generic.go:356] "Generic (PLEG): container finished" podID="8e313269-b052-4ca0-8348-8167b5fe0059" containerID="aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f" exitCode=0 Jul 08 11:47:19.729141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.728994 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" Jul 08 11:47:19.729141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.729001 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerDied","Data":"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f"} Jul 08 11:47:19.729141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.729078 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h" event={"ID":"8e313269-b052-4ca0-8348-8167b5fe0059","Type":"ContainerDied","Data":"b5afe32ccdb07aea040e3a2304297a650891a5a70ba5e5121e3edd5af0aaf4e4"} Jul 08 11:47:19.729141 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.729109 2559 scope.go:117] "RemoveContainer" containerID="aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f" Jul 08 11:47:19.739546 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.739527 2559 scope.go:117] "RemoveContainer" containerID="221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa" Jul 08 11:47:19.748541 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.748521 2559 scope.go:117] "RemoveContainer" containerID="aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f" Jul 08 11:47:19.748829 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:47:19.748809 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f\": container with ID starting with aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f not found: ID does not exist" containerID="aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f" Jul 08 11:47:19.748879 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.748840 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f"} err="failed to get container status \"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f\": rpc error: code = NotFound desc = could not find container \"aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f\": container with ID starting with aa69875b89f53f2dfae1a59e7069f072a3dfcd7c039e2815ae6a701781e2840f not found: ID does not exist" Jul 08 11:47:19.748879 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.748866 2559 scope.go:117] "RemoveContainer" containerID="221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa" Jul 08 11:47:19.749128 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:47:19.749110 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa\": container with ID starting with 221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa not found: ID does not exist" containerID="221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa" Jul 08 11:47:19.749165 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.749135 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa"} err="failed to get container status \"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa\": rpc error: code = NotFound desc = could not find container \"221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa\": container with ID starting with 221a1360c60420edbcad552711e883b9d57c7baa52e417a19d964a9ae85622fa not found: ID does not exist" Jul 08 11:47:19.753891 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.753869 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:47:19.759090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:19.759069 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5cec6d9a-openshift-default-577b98d887-tls2h"] Jul 08 11:47:20.145108 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:20.145022 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:20.169360 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:20.169330 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" path="/var/lib/kubelet/pods/8e313269-b052-4ca0-8348-8167b5fe0059/volumes" Jul 08 11:47:23.502213 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:23.502179 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:23.569137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:23.569101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:24.185493 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:24.185457 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:30.097049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:30.096959 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:30.203084 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:30.203044 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:30.466064 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:30.466026 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:40.793760 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:40.793727 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:47:42.874550 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:47:42.874517 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:02.094007 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:02.093960 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:04.277524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:04.277492 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:48:04.280245 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:04.280226 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:48:07.886847 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:07.886811 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:07.986929 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:07.986899 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:08.267174 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.267137 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:08.552739 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.552628 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:48:08.553101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553086 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.553149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553105 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.553149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553115 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.553149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553122 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.553149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553135 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" containerName="main" Jul 08 11:48:08.553149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553141 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" containerName="main" Jul 08 11:48:08.553331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553242 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.553331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553254 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70d3946b-cd74-4dba-bf16-d25a6b064b7a" containerName="main" Jul 08 11:48:08.553331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.553260 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e313269-b052-4ca0-8348-8167b5fe0059" containerName="istio-proxy" Jul 08 11:48:08.556537 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.556509 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.560175 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.560152 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3ac51b-openshift-default-dockercfg-nvqml\"" Jul 08 11:48:08.573045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.573017 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:48:08.581191 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.581161 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:08.676802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.676765 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.676802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.676802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.676835 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsbvp\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.676944 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.676976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.677002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677182 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.677065 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677182 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.677081 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.677182 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.677098 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778086 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778051 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778179 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778199 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778309 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778256 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778318 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsbvp\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778623 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778793 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.778959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.778940 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.779055 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.779035 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.780933 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.780911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.781107 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.781090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.786850 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.786822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.786988 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.786895 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsbvp\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp\") pod \"e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:08.868510 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:08.868363 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:09.038378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:09.037927 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:48:09.040563 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:09.040361 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:09.040563 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:09.040466 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:09.040563 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:09.040503 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:09.370918 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:09.370889 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:10.021290 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:10.021256 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerStarted","Data":"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0"} Jul 08 11:48:10.021449 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:10.021331 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerStarted","Data":"70c8fc9e226af58ab18b766b49f9f3bcf9f1446be5564888969241ed31abca57"} Jul 08 11:48:10.045061 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:10.044998 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podStartSLOduration=2.044971981 podStartE2EDuration="2.044971981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:48:10.043550493 +0000 UTC m=+2106.425197262" watchObservedRunningTime="2026-07-08 11:48:10.044971981 +0000 UTC m=+2106.426618755" Jul 08 11:48:10.868828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:10.868792 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:10.873970 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:10.873938 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:11.027025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:11.026992 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:11.028033 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:11.028011 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:12.023859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.023827 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:12.132863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.132789 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:12.221867 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.221822 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:12.414916 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.414827 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:12.803610 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.803582 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:12.904302 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:12.904271 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:16.531211 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:16.531178 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:16.851055 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:16.850974 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:17.171180 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.171104 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:48:17.175678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.175654 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.179139 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.179120 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ae9da2-openshift-default-dockercfg-kpdjj\"" Jul 08 11:48:17.187217 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.187192 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:48:17.240161 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.240128 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:17.356294 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.356259 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:17.359277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359412 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359412 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359344 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359519 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359477 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359573 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359525 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvr5\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359657 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359578 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359657 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359746 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.359797 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.359772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461090 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461056 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461112 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461357 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461524 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvr5\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461737 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461579 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.461737 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461731 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.462017 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.462017 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461984 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.462017 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.461988 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.463738 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.463713 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.464024 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.464006 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.470472 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.470450 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.470729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.470706 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvr5\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5\") pod \"e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.487569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.487537 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:17.638131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.638104 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:48:17.641858 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:48:17.641824 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173f22b9_c591_4377_bbca_a2c14d6e1b70.slice/crio-9128af5eee58ed34b15a9ad2d6c7ee6d89b9f34b74f650a1dc86257528d98480 WatchSource:0}: Error finding container 9128af5eee58ed34b15a9ad2d6c7ee6d89b9f34b74f650a1dc86257528d98480: Status 404 returned error can't find the container with id 9128af5eee58ed34b15a9ad2d6c7ee6d89b9f34b74f650a1dc86257528d98480 Jul 08 11:48:17.644195 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.644160 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:17.644296 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.644227 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:17.644296 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:17.644255 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:18.067907 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:18.067874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerStarted","Data":"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6"} Jul 08 11:48:18.068102 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:18.067928 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerStarted","Data":"9128af5eee58ed34b15a9ad2d6c7ee6d89b9f34b74f650a1dc86257528d98480"} Jul 08 11:48:18.089072 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:18.089018 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podStartSLOduration=1.089001454 podStartE2EDuration="1.089001454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:48:18.085931464 +0000 UTC m=+2114.467578246" watchObservedRunningTime="2026-07-08 11:48:18.089001454 +0000 UTC m=+2114.470648224" Jul 08 11:48:18.488476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:18.488441 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:18.493553 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:18.493525 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:19.073939 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:19.073907 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:19.074994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:19.074976 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:20.016079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:20.016047 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:20.126854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:20.126816 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:20.348796 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:20.348686 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:21.090168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:21.082987 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:21.183739 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:21.183695 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:21.280834 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:21.280800 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:24.531871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:24.531837 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:27.068959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:27.068916 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:48:27.069430 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:27.069007 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:48:27.095222 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:27.095183 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:29.142472 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.142375 2559 generic.go:356] "Generic (PLEG): container finished" podID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerID="a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0" exitCode=0 Jul 08 11:48:29.142472 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.142453 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerDied","Data":"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0"} Jul 08 11:48:29.143217 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.142926 2559 scope.go:117] "RemoveContainer" containerID="a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0" Jul 08 11:48:29.144045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.144012 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:29.144140 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.144081 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:29.144140 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:29.144128 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:30.150622 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:30.150585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerStarted","Data":"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd"} Jul 08 11:48:30.869186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:30.869152 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:30.869388 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:30.869369 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 11:48:30.869447 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:30.869417 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 11:48:31.869723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:31.869671 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 11:48:31.870327 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:31.870292 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 11:48:32.869158 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:32.869114 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 11:48:32.869424 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:32.869211 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 11:48:33.811964 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:33.811868 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:33.869516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:33.869467 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 08 11:48:33.869747 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:33.869547 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 08 11:48:33.911587 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:33.911544 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:34.873006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:34.872975 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:34.873388 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:34.873254 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:34.873886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:34.873863 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:48:39.226479 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.226444 2559 generic.go:356] "Generic (PLEG): container finished" podID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerID="f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6" exitCode=0 Jul 08 11:48:39.226911 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.226515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerDied","Data":"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6"} Jul 08 11:48:39.226957 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.226937 2559 scope.go:117] "RemoveContainer" containerID="f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6" Jul 08 11:48:39.228079 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.228054 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:39.228134 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.228119 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:39.228168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:39.228155 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:48:40.233586 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:40.233552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerStarted","Data":"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea"} Jul 08 11:48:40.488131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:40.488040 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:40.488287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:40.488231 2559 patch_prober.go:28] interesting pod/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 11:48:40.488344 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:40.488281 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 11:48:41.488984 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:41.488952 2559 patch_prober.go:28] interesting pod/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 11:48:41.489510 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:41.489016 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 11:48:41.889722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:41.889599 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:41.990401 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:41.990368 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:48:42.488865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:42.488773 2559 patch_prober.go:28] interesting pod/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 11:48:42.488865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:42.488828 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 11:48:43.488110 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:43.488075 2559 patch_prober.go:28] interesting pod/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 08 11:48:43.488326 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:43.488145 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 08 11:48:44.491914 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:44.491886 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:44.492285 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:44.492229 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:44.492852 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:44.492832 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:48:52.459466 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:48:52.459433 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:12.901152 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:12.901109 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:13.792640 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:13.792602 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:13.955041 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:13.955007 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:14.379895 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:14.379859 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:14.476154 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:14.476120 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:14.577491 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:14.577442 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:14.688979 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:14.688886 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:19.554263 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:19.554230 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:43.613771 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:43.613735 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:45.839441 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:45.839412 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:46.081112 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:46.081077 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:49:46.081345 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:46.081319 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" containerID="cri-o://a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd" gracePeriod=30 Jul 08 11:49:46.105865 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:46.105787 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:46.511915 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:46.511868 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:46.983035 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:46.983006 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:47.071587 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:47.071537 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:47.173313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:47.173264 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:47.308948 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:47.308868 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:49.875583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:49.875539 2559 patch_prober.go:28] interesting pod/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 11:49:49.875973 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:49.875618 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 11:49:51.347984 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.347920 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:49:51.357118 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357097 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357128 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357154 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357184 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357231 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsbvp\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357263 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357298 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357335 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357365 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket\") pod \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\" (UID: \"a1030a92-c9e1-4ab7-b075-b39de9619cbd\") " Jul 08 11:49:51.357660 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357522 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:51.357756 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357729 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.357756 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357733 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:51.357756 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357725 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:51.358034 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.357984 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:49:51.358648 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.358608 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:51.359672 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.359626 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:49:51.359767 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.359702 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp" (OuterVolumeSpecName: "kube-api-access-xsbvp") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "kube-api-access-xsbvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:49:51.359898 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.359880 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:49:51.359952 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.359938 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1030a92-c9e1-4ab7-b075-b39de9619cbd" (UID: "a1030a92-c9e1-4ab7-b075-b39de9619cbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:51.458404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458367 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458399 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458404 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458409 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458418 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458428 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458436 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsbvp\" (UniqueName: \"kubernetes.io/projected/a1030a92-c9e1-4ab7-b075-b39de9619cbd-kube-api-access-xsbvp\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458445 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.458680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.458453 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1030a92-c9e1-4ab7-b075-b39de9619cbd-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:51.663138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.663047 2559 generic.go:356] "Generic (PLEG): container finished" podID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerID="a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd" exitCode=0 Jul 08 11:49:51.663138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.663101 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" Jul 08 11:49:51.663138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.663113 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerDied","Data":"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd"} Jul 08 11:49:51.663385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.663166 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp" event={"ID":"a1030a92-c9e1-4ab7-b075-b39de9619cbd","Type":"ContainerDied","Data":"70c8fc9e226af58ab18b766b49f9f3bcf9f1446be5564888969241ed31abca57"} Jul 08 11:49:51.663385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.663196 2559 scope.go:117] "RemoveContainer" containerID="a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd" Jul 08 11:49:51.673353 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.673331 2559 scope.go:117] "RemoveContainer" containerID="a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0" Jul 08 11:49:51.681989 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.681967 2559 scope.go:117] "RemoveContainer" containerID="a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd" Jul 08 11:49:51.682536 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:49:51.682482 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd\": container with ID starting with a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd not found: ID does not exist" containerID="a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd" Jul 08 11:49:51.682536 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.682522 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd"} err="failed to get container status \"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd\": rpc error: code = NotFound desc = could not find container \"a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd\": container with ID starting with a653db703645e8be326ac00e20469a8a38c2ef412c1d5c57c1436cb6a7c205bd not found: ID does not exist" Jul 08 11:49:51.682739 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.682547 2559 scope.go:117] "RemoveContainer" containerID="a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0" Jul 08 11:49:51.682955 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:49:51.682932 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0\": container with ID starting with a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0 not found: ID does not exist" containerID="a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0" Jul 08 11:49:51.683043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.682966 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0"} err="failed to get container status \"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0\": rpc error: code = NotFound desc = could not find container \"a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0\": container with ID starting with a117ac0ae9b115692b65078b8242d9973a844c98c282cb67ebacf727623479f0 not found: ID does not exist" Jul 08 11:49:51.685048 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.685029 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:49:51.689982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:51.689958 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3ac51b-openshift-default-648c8b7ff5-zshkp"] Jul 08 11:49:52.170575 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:52.170539 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" path="/var/lib/kubelet/pods/a1030a92-c9e1-4ab7-b075-b39de9619cbd/volumes" Jul 08 11:49:52.811605 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:52.811567 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:52.912107 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:52.912076 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:53.058012 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.057981 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:49:53.058244 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.058220 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" containerID="cri-o://f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea" gracePeriod=30 Jul 08 11:49:53.075260 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.075181 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:53.649837 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.649797 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:53.943109 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943020 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:49:53.943617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943599 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.943702 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943621 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.943702 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943674 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.943702 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943684 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.943854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943795 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.943854 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.943808 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1030a92-c9e1-4ab7-b075-b39de9619cbd" containerName="istio-proxy" Jul 08 11:49:53.948646 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.948606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.951341 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.951321 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4076ef-openshift-default-dockercfg-89gfx\"" Jul 08 11:49:53.960352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.960310 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:49:53.970824 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.970788 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.980967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981105 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981138 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7zk\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981187 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981248 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:53.981421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:53.981307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.081949 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.081912 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.081961 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.081981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082150 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082110 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082298 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7zk\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082298 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082210 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082298 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082419 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082419 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082368 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082485 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082368 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082773 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082848 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.082919 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.082895 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.083356 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.083284 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.084938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.084909 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.085255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.085235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.090451 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.090430 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7zk\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.091119 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.091099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token\") pod \"e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.186968 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.186936 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:54.259803 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.259762 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:54.410287 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.410258 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:49:54.412979 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:49:54.412954 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45bfaa23_31eb_41f5_9425_eb8ba9b06d4e.slice/crio-ffa3f106917d2b6aac601b05534b9a5aeff3446ebc2e38fd9551a8f48e5113c0 WatchSource:0}: Error finding container ffa3f106917d2b6aac601b05534b9a5aeff3446ebc2e38fd9551a8f48e5113c0: Status 404 returned error can't find the container with id ffa3f106917d2b6aac601b05534b9a5aeff3446ebc2e38fd9551a8f48e5113c0 Jul 08 11:49:54.415098 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.415064 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:49:54.415204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.415141 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:49:54.415204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.415183 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:49:54.693321 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.693225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerStarted","Data":"789ae68ee83370b409f3150a8262de00da4d1ff7481de41c3690bcee28f124b1"} Jul 08 11:49:54.693321 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.693298 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerStarted","Data":"ffa3f106917d2b6aac601b05534b9a5aeff3446ebc2e38fd9551a8f48e5113c0"} Jul 08 11:49:54.714138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:54.714085 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podStartSLOduration=1.714068121 podStartE2EDuration="1.714068121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:49:54.712992163 +0000 UTC m=+2211.094638934" watchObservedRunningTime="2026-07-08 11:49:54.714068121 +0000 UTC m=+2211.095714891" Jul 08 11:49:55.260076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:55.260035 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:55.264935 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:55.264908 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:55.698913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:55.698837 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:55.699812 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:55.699785 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:49:56.632306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:56.632268 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:56.727928 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:56.727888 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:56.804391 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:56.804360 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:57.215342 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:57.215305 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:57.313199 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:57.313160 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:57.382366 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:57.382329 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:49:58.325698 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.325617 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:49:58.420860 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.420812 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.420871 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nvr5\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.420901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.420973 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421030 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421056 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421050 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421123 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421183 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421231 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:58.421325 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421239 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token\") pod \"173f22b9-c591-4377-bbca-a2c14d6e1b70\" (UID: \"173f22b9-c591-4377-bbca-a2c14d6e1b70\") " Jul 08 11:49:58.421522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421420 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:58.421522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421342 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:49:58.421624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421595 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:58.421731 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421711 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.421789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421738 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.421789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421755 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f22b9-c591-4377-bbca-a2c14d6e1b70-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.421789 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421769 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.422018 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.421995 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data" (OuterVolumeSpecName: "istio-data") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:58.423432 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.423398 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:49:58.423539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.423485 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5" (OuterVolumeSpecName: "kube-api-access-9nvr5") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "kube-api-access-9nvr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:49:58.423824 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.423807 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token" (OuterVolumeSpecName: "istio-token") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:49:58.423883 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.423823 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "173f22b9-c591-4377-bbca-a2c14d6e1b70" (UID: "173f22b9-c591-4377-bbca-a2c14d6e1b70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:49:58.522982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.522941 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.522982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.522973 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.522982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.522982 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.522982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.522991 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f22b9-c591-4377-bbca-a2c14d6e1b70-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.523251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.523000 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nvr5\" (UniqueName: \"kubernetes.io/projected/173f22b9-c591-4377-bbca-a2c14d6e1b70-kube-api-access-9nvr5\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:49:58.719910 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.719875 2559 generic.go:356] "Generic (PLEG): container finished" podID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerID="f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea" exitCode=0 Jul 08 11:49:58.720115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.719931 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" Jul 08 11:49:58.720115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.719958 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerDied","Data":"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea"} Jul 08 11:49:58.720115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.720021 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz" event={"ID":"173f22b9-c591-4377-bbca-a2c14d6e1b70","Type":"ContainerDied","Data":"9128af5eee58ed34b15a9ad2d6c7ee6d89b9f34b74f650a1dc86257528d98480"} Jul 08 11:49:58.720115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.720051 2559 scope.go:117] "RemoveContainer" containerID="f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea" Jul 08 11:49:58.730108 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.730093 2559 scope.go:117] "RemoveContainer" containerID="f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6" Jul 08 11:49:58.738442 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.738424 2559 scope.go:117] "RemoveContainer" containerID="f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea" Jul 08 11:49:58.738876 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:49:58.738852 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea\": container with ID starting with f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea not found: ID does not exist" containerID="f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea" Jul 08 11:49:58.739155 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.738882 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea"} err="failed to get container status \"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea\": rpc error: code = NotFound desc = could not find container \"f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea\": container with ID starting with f1aab68a458e6b40e9f0f5b9805dd486d5ab61ee77df6baf32661132e52f13ea not found: ID does not exist" Jul 08 11:49:58.739155 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.738901 2559 scope.go:117] "RemoveContainer" containerID="f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6" Jul 08 11:49:58.739275 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:49:58.739153 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6\": container with ID starting with f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6 not found: ID does not exist" containerID="f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6" Jul 08 11:49:58.739275 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.739178 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6"} err="failed to get container status \"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6\": rpc error: code = NotFound desc = could not find container \"f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6\": container with ID starting with f2b28cb7fde05f8ddb688f9f4914089f0316452c2a5dc45c21517098cf4dd1f6 not found: ID does not exist" Jul 08 11:49:58.743952 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.743916 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:49:58.750670 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:49:58.750614 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ae9da2-openshift-default-7b45575c5f-2c6fz"] Jul 08 11:50:00.169939 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:00.169904 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" path="/var/lib/kubelet/pods/173f22b9-c591-4377-bbca-a2c14d6e1b70/volumes" Jul 08 11:50:01.145352 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.145318 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:01.833546 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.833507 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:01.961396 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.961364 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:01.961839 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.961825 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:01.961963 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.961840 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:01.961963 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.961916 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:01.966880 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.966858 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:01.969569 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.969548 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-cf8404-openshift-default-dockercfg-4hnxk\"" Jul 08 11:50:01.980436 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.980404 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:01.991091 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:01.991058 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:02.057087 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057087 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057089 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds5xl\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057132 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057380 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057374 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.057755 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.057463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158605 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158605 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158555 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158605 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158614 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds5xl\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158726 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.158909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.159236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.158996 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.159236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.159011 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.159236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.159102 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.159236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.159185 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.159843 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.159822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.160980 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.160958 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.161427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.161404 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.167727 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.167690 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.167869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.167850 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds5xl\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl\") pod \"e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.197236 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.197190 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:02.277526 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.277484 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:02.438973 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.438934 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:02.439577 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:50:02.439549 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75ce6f8_a834_483e_bf8b_16de19fa0d1c.slice/crio-0d0f2b097f9d749c2b897de7d55aa5b5fbad877fbcc7ace22bff18904b5ceb23 WatchSource:0}: Error finding container 0d0f2b097f9d749c2b897de7d55aa5b5fbad877fbcc7ace22bff18904b5ceb23: Status 404 returned error can't find the container with id 0d0f2b097f9d749c2b897de7d55aa5b5fbad877fbcc7ace22bff18904b5ceb23 Jul 08 11:50:02.442081 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.442053 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:02.442158 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.442117 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:02.442158 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.442146 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:02.751318 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.751287 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerStarted","Data":"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c"} Jul 08 11:50:02.751479 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.751341 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerStarted","Data":"0d0f2b097f9d749c2b897de7d55aa5b5fbad877fbcc7ace22bff18904b5ceb23"} Jul 08 11:50:02.776794 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:02.776737 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podStartSLOduration=1.776715888 podStartE2EDuration="1.776715888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:50:02.773688619 +0000 UTC m=+2219.155335380" watchObservedRunningTime="2026-07-08 11:50:02.776715888 +0000 UTC m=+2219.158362659" Jul 08 11:50:03.278170 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:03.278132 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:03.283314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:03.283289 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:03.756849 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:03.756818 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:03.757938 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:03.757920 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:04.809824 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:04.809795 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:04.905200 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:04.905166 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:05.042650 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:05.042578 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:05.229390 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:05.229356 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:05.824729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:05.824686 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:05.949540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:05.949504 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:09.224825 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:09.224794 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:11.742831 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:11.742783 2559 patch_prober.go:28] interesting pod/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:50:11.743204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:11.742868 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:50:12.810647 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.810601 2559 generic.go:356] "Generic (PLEG): container finished" podID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerID="789ae68ee83370b409f3150a8262de00da4d1ff7481de41c3690bcee28f124b1" exitCode=0 Jul 08 11:50:12.811063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.810680 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerDied","Data":"789ae68ee83370b409f3150a8262de00da4d1ff7481de41c3690bcee28f124b1"} Jul 08 11:50:12.811116 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.811101 2559 scope.go:117] "RemoveContainer" containerID="789ae68ee83370b409f3150a8262de00da4d1ff7481de41c3690bcee28f124b1" Jul 08 11:50:12.812210 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.812185 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:12.812267 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.812252 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:12.812313 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:12.812279 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:13.818176 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:13.818141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerStarted","Data":"1d3d5de5fc6aa63fa143a5680e4269ceb86b450663e0c8190c33ccad3c45bf59"} Jul 08 11:50:14.260954 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:14.260913 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:14.261266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:14.261047 2559 patch_prober.go:28] interesting pod/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 11:50:14.261266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:14.261092 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 11:50:15.261099 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:15.261054 2559 patch_prober.go:28] interesting pod/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 11:50:15.261624 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:15.261141 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 11:50:16.260509 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:16.260467 2559 patch_prober.go:28] interesting pod/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 11:50:16.260869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:16.260829 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 11:50:17.260999 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.260960 2559 patch_prober.go:28] interesting pod/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 08 11:50:17.261611 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.261578 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 08 11:50:17.376745 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.376713 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:17.764294 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.764245 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:17.867910 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.867877 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:17.966646 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:17.966531 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:18.261070 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:18.260986 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:18.263782 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:18.263749 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:18.264166 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:18.264142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:18.264886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:18.264866 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:19.758087 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:19.758043 2559 patch_prober.go:28] interesting pod/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:50:19.758513 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:19.758119 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:50:20.864060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.864028 2559 generic.go:356] "Generic (PLEG): container finished" podID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerID="42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c" exitCode=0 Jul 08 11:50:20.864573 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.864101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerDied","Data":"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c"} Jul 08 11:50:20.864573 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.864524 2559 scope.go:117] "RemoveContainer" containerID="42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c" Jul 08 11:50:20.865531 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.865501 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:20.865614 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.865596 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:20.865680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:20.865663 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:21.871678 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:21.871621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerStarted","Data":"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a"} Jul 08 11:50:22.278320 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:22.278274 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:22.278542 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:22.278412 2559 patch_prober.go:28] interesting pod/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 11:50:22.278542 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:22.278493 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 11:50:23.277959 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.277914 2559 patch_prober.go:28] interesting pod/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 11:50:23.278442 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.277992 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 11:50:23.553680 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.553557 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:23.620531 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.620504 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:23.789961 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.789919 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:50:23.790214 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.790190 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" containerID="cri-o://1d3d5de5fc6aa63fa143a5680e4269ceb86b450663e0c8190c33ccad3c45bf59" gracePeriod=30 Jul 08 11:50:23.815300 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:23.815186 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:24.278458 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.278415 2559 patch_prober.go:28] interesting pod/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 11:50:24.278952 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.278481 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 11:50:24.347168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.347124 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:24.763434 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.763402 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:24.863435 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.863348 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:24.963857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:24.963822 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:25.110455 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:25.110370 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:25.278832 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:25.278799 2559 patch_prober.go:28] interesting pod/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 08 11:50:25.279292 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:25.278858 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 08 11:50:26.281708 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:26.281681 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:26.282101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:26.282054 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:26.282752 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:26.282734 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:28.917714 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:28.917683 2559 generic.go:356] "Generic (PLEG): container finished" podID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerID="1d3d5de5fc6aa63fa143a5680e4269ceb86b450663e0c8190c33ccad3c45bf59" exitCode=0 Jul 08 11:50:28.918093 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:28.917756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerDied","Data":"1d3d5de5fc6aa63fa143a5680e4269ceb86b450663e0c8190c33ccad3c45bf59"} Jul 08 11:50:28.918093 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:28.917845 2559 scope.go:117] "RemoveContainer" containerID="789ae68ee83370b409f3150a8262de00da4d1ff7481de41c3690bcee28f124b1" Jul 08 11:50:29.051973 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.051909 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:29.131560 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131522 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131581 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131618 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131684 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131707 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm7zk\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131730 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131750 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.131786 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131772 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.132153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131803 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token\") pod \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\" (UID: \"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e\") " Jul 08 11:50:29.132153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.131914 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:29.132153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.132096 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.132153 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.132110 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:29.132610 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.132354 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:29.132610 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.132449 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:50:29.132913 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.132882 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data" (OuterVolumeSpecName: "istio-data") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:29.134135 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.134106 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:50:29.134230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.134133 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk" (OuterVolumeSpecName: "kube-api-access-gm7zk") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "kube-api-access-gm7zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:50:29.134230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.134180 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token" (OuterVolumeSpecName: "istio-token") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:50:29.134230 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.134184 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" (UID: "45bfaa23-31eb-41f5-9425-eb8ba9b06d4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:29.232661 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232600 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232661 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232629 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm7zk\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-kube-api-access-gm7zk\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232672 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232684 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232696 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232707 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232718 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.232864 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.232728 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:29.925092 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.925066 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" Jul 08 11:50:29.925590 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.925109 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp" event={"ID":"45bfaa23-31eb-41f5-9425-eb8ba9b06d4e","Type":"ContainerDied","Data":"ffa3f106917d2b6aac601b05534b9a5aeff3446ebc2e38fd9551a8f48e5113c0"} Jul 08 11:50:29.925590 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.925169 2559 scope.go:117] "RemoveContainer" containerID="1d3d5de5fc6aa63fa143a5680e4269ceb86b450663e0c8190c33ccad3c45bf59" Jul 08 11:50:29.947811 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.947778 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:50:29.954068 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:29.954044 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4076ef-openshift-default-68b454bb8c-pgxvp"] Jul 08 11:50:30.169844 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.169807 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" path="/var/lib/kubelet/pods/45bfaa23-31eb-41f5-9425-eb8ba9b06d4e/volumes" Jul 08 11:50:30.591520 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.591492 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:30.662500 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.662457 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:30.857179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.857095 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:30.857321 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.857303 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" containerID="cri-o://4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a" gracePeriod=30 Jul 08 11:50:30.879358 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:30.879329 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:31.421268 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.421229 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:31.715263 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715233 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:50:31.715654 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715625 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:31.715734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715657 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:31.715734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715682 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:31.715734 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715688 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:31.715881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715761 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="173f22b9-c591-4377-bbca-a2c14d6e1b70" containerName="istio-proxy" Jul 08 11:50:31.715881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715769 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:31.715881 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.715776 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:31.720577 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.720554 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.723286 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.723266 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-5a670c-openshift-default-dockercfg-wvksv\"" Jul 08 11:50:31.732116 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.732091 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:50:31.748808 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.748777 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:31.757077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757023 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757385 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757583 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcfkm\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757772 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757790 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757824 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757995 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.757995 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.757970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.758083 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.758004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.858982 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.858952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.858988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfkm\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859129 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859193 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859412 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859709 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859538 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859766 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859736 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859847 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859821 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.859910 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.859835 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.861768 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.861748 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.861878 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.861811 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.867440 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.867413 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:31.867896 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:31.867880 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcfkm\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm\") pod \"e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:32.032200 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.032095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:32.190037 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.190011 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:50:32.192224 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:50:32.192193 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c046e5_6b68_4ace_ba4f_c9d5680daf1a.slice/crio-24c3fd0aecb38684679779fdb7034ed98d6895fcbfd471326e87adb5bdef7956 WatchSource:0}: Error finding container 24c3fd0aecb38684679779fdb7034ed98d6895fcbfd471326e87adb5bdef7956: Status 404 returned error can't find the container with id 24c3fd0aecb38684679779fdb7034ed98d6895fcbfd471326e87adb5bdef7956 Jul 08 11:50:32.194389 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.194358 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:32.194486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.194427 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:32.194486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.194457 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:32.311599 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.311510 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:32.945960 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.945914 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerStarted","Data":"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286"} Jul 08 11:50:32.946323 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.946021 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerStarted","Data":"24c3fd0aecb38684679779fdb7034ed98d6895fcbfd471326e87adb5bdef7956"} Jul 08 11:50:32.967251 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:32.967198 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podStartSLOduration=1.967182395 podStartE2EDuration="1.967182395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:50:32.964835575 +0000 UTC m=+2249.346482348" watchObservedRunningTime="2026-07-08 11:50:32.967182395 +0000 UTC m=+2249.348829164" Jul 08 11:50:33.032445 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:33.032403 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:33.038120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:33.038091 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:33.951340 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:33.951308 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:33.952223 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:33.952208 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:34.972863 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:34.972826 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:35.061920 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:35.061871 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:35.181225 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:35.181186 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:35.385782 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:35.385692 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:35.677780 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:35.677550 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:35.772869 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:35.772832 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:36.108985 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.108962 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:36.198225 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198187 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198236 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198261 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198297 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds5xl\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198393 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198340 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198449 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198491 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198521 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198620 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198548 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs\") pod \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\" (UID: \"b75ce6f8-a834-483e-bf8b-16de19fa0d1c\") " Jul 08 11:50:36.198830 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198612 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:36.198830 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198627 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:50:36.198931 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198907 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:36.198931 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198921 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:36.199025 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.198968 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.199151 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.199057 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.199746 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.199722 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:36.200801 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.200776 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl" (OuterVolumeSpecName: "kube-api-access-ds5xl") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "kube-api-access-ds5xl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:50:36.200978 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.200951 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:50:36.201246 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.201226 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:50:36.201311 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.201223 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b75ce6f8-a834-483e-bf8b-16de19fa0d1c" (UID: "b75ce6f8-a834-483e-bf8b-16de19fa0d1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300039 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300076 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds5xl\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-kube-api-access-ds5xl\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300089 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300101 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300114 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300124 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.300138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.300136 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b75ce6f8-a834-483e-bf8b-16de19fa0d1c-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:50:36.971304 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.971265 2559 generic.go:356] "Generic (PLEG): container finished" podID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerID="4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a" exitCode=0 Jul 08 11:50:36.971453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.971313 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerDied","Data":"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a"} Jul 08 11:50:36.971453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.971331 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" Jul 08 11:50:36.971453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.971400 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng" event={"ID":"b75ce6f8-a834-483e-bf8b-16de19fa0d1c","Type":"ContainerDied","Data":"0d0f2b097f9d749c2b897de7d55aa5b5fbad877fbcc7ace22bff18904b5ceb23"} Jul 08 11:50:36.971453 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.971423 2559 scope.go:117] "RemoveContainer" containerID="4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a" Jul 08 11:50:36.981095 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.981077 2559 scope.go:117] "RemoveContainer" containerID="42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c" Jul 08 11:50:36.989572 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.989556 2559 scope.go:117] "RemoveContainer" containerID="4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a" Jul 08 11:50:36.989814 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:50:36.989796 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a\": container with ID starting with 4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a not found: ID does not exist" containerID="4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a" Jul 08 11:50:36.989882 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.989821 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a"} err="failed to get container status \"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a\": rpc error: code = NotFound desc = could not find container \"4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a\": container with ID starting with 4d24b64a4e0c0eb606168a0b08e26945a8475d5a4be93b9c825fdcc4b243c65a not found: ID does not exist" Jul 08 11:50:36.989882 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.989838 2559 scope.go:117] "RemoveContainer" containerID="42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c" Jul 08 11:50:36.990078 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:50:36.990062 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c\": container with ID starting with 42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c not found: ID does not exist" containerID="42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c" Jul 08 11:50:36.990114 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.990084 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c"} err="failed to get container status \"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c\": rpc error: code = NotFound desc = could not find container \"42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c\": container with ID starting with 42f336d4127f82a1442983290e95c161992accdb09f0fd140c54f0f4dcb2760c not found: ID does not exist" Jul 08 11:50:36.995159 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:36.995134 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:37.001266 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:37.001245 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-cf8404-openshift-default-6fd59cb6c5-8qwng"] Jul 08 11:50:38.170199 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:38.170164 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" path="/var/lib/kubelet/pods/b75ce6f8-a834-483e-bf8b-16de19fa0d1c/volumes" Jul 08 11:50:39.377054 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:39.377021 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:39.724363 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:39.724312 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.002885 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003759 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003780 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003846 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003855 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="45bfaa23-31eb-41f5-9425-eb8ba9b06d4e" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003871 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.003893 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.004080 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.004722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.004101 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75ce6f8-a834-483e-bf8b-16de19fa0d1c" containerName="istio-proxy" Jul 08 11:50:40.010071 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.010049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.015494 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.015415 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e79b1c-openshift-default-dockercfg-nlpb5\"" Jul 08 11:50:40.018521 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.018491 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:50:40.042575 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.042530 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:40.134613 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134577 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.134845 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.134845 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twnmf\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.134965 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.134965 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134922 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.134965 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.134958 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.135107 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.135000 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.135107 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.135025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.135107 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.135078 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.138793 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.138764 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:40.236486 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twnmf\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236549 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236653 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.236693 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.236951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237015 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237085 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237224 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237214 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237355 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237326 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.237511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.237359 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.239428 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.239405 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.239695 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.239677 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.245211 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.245181 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.245476 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.245457 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twnmf\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf\") pod \"e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.322512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.322422 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:40.469249 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.469225 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:50:40.470852 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:50:40.470826 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a10f2cd_df92_4234_a11e_80b998b93c5b.slice/crio-635ed0ccc60558c22e8b5ec6380c068c128b2130220f476797a1a4450e8e38ee WatchSource:0}: Error finding container 635ed0ccc60558c22e8b5ec6380c068c128b2130220f476797a1a4450e8e38ee: Status 404 returned error can't find the container with id 635ed0ccc60558c22e8b5ec6380c068c128b2130220f476797a1a4450e8e38ee Jul 08 11:50:40.473017 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.472964 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:40.473129 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.473054 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:40.473129 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.473092 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:40.999004 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.998971 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerStarted","Data":"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5"} Jul 08 11:50:40.999155 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:40.999019 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerStarted","Data":"635ed0ccc60558c22e8b5ec6380c068c128b2130220f476797a1a4450e8e38ee"} Jul 08 11:50:41.022781 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:41.022731 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podStartSLOduration=2.022714519 podStartE2EDuration="2.022714519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 11:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:50:41.02005865 +0000 UTC m=+2257.401705422" watchObservedRunningTime="2026-07-08 11:50:41.022714519 +0000 UTC m=+2257.404361339" Jul 08 11:50:41.323312 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:41.323207 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:41.328215 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:41.328187 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:42.006370 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:42.006343 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:42.007277 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:42.007253 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:50:42.920066 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:42.920033 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:43.004833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:43.004792 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:43.075183 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:43.075141 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:43.508809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:43.508778 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:43.632093 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:43.632061 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:43.731164 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:43.731111 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:47.443422 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:47.443381 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:49.574272 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:49.574234 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:49.675449 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:49.675406 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:49.993835 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:49.993794 2559 patch_prober.go:28] interesting pod/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:50:49.994019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:49.993873 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:50:51.064064 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.064031 2559 generic.go:356] "Generic (PLEG): container finished" podID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerID="50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286" exitCode=0 Jul 08 11:50:51.064547 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.064081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerDied","Data":"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286"} Jul 08 11:50:51.064658 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.064619 2559 scope.go:117] "RemoveContainer" containerID="50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286" Jul 08 11:50:51.065730 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.065699 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:51.065802 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.065768 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:51.065843 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:51.065798 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:50:52.076722 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:52.076689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerStarted","Data":"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597"} Jul 08 11:50:53.033001 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:53.032959 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:53.033306 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:53.033282 2559 patch_prober.go:28] interesting pod/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 11:50:53.033416 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:53.033340 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 11:50:54.032662 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:54.032588 2559 patch_prober.go:28] interesting pod/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 11:50:54.033168 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:54.032694 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 11:50:55.032517 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:55.032488 2559 patch_prober.go:28] interesting pod/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 08 11:50:55.032763 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:55.032734 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 08 11:50:56.042173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:56.040845 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:56.042173 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:56.042117 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:56.043606 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:56.043587 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:50:56.535760 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:56.535718 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:56.636928 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:56.636829 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:50:58.048873 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:58.048828 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 11:50:58.049378 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:50:58.048897 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 11:51:00.135039 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.135004 2559 generic.go:356] "Generic (PLEG): container finished" podID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerID="14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5" exitCode=0 Jul 08 11:51:00.135474 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.135081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerDied","Data":"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5"} Jul 08 11:51:00.135551 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.135535 2559 scope.go:117] "RemoveContainer" containerID="14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5" Jul 08 11:51:00.136750 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.136720 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:51:00.136859 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.136808 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:51:00.136929 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:00.136861 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 08 11:51:01.152179 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:01.152147 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerStarted","Data":"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916"} Jul 08 11:51:01.323522 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:01.323481 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:51:01.323721 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:01.323703 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 08 11:51:01.323779 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:01.323761 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 08 11:51:02.323354 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:02.323316 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 08 11:51:02.323818 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:02.323385 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 08 11:51:03.323691 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:03.323618 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 08 11:51:03.324177 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:03.323716 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 08 11:51:04.323350 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:04.323311 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 08 11:51:04.323539 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:04.323382 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 08 11:51:04.387186 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:04.387150 2559 scope.go:117] "RemoveContainer" containerID="b1c821a075ff303e1728c26863f2fbb6be6b2a0ac9469cb1c63a93e8b9d54a8b" Jul 08 11:51:04.613406 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:04.613309 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:51:04.721338 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:04.721307 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:51:05.323808 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:05.323772 2559 patch_prober.go:28] interesting pod/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 08 11:51:05.323978 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:05.323830 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 08 11:51:06.327121 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:06.327093 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:51:06.327582 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:06.327344 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:51:06.328059 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:06.328043 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:51:14.335874 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:14.335829 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:51:42.605118 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:51:42.605032 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:52:04.310060 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:52:04.310019 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:52:04.512858 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:52:04.512823 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:52:04.600026 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:52:04.599923 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:04.158652 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:04.158600 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:04.318293 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:04.318264 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:53:04.322564 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:04.322541 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:53:52.580958 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:52.580918 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:52.995124 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:52.995077 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:53.183967 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:53.183933 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:53.365043 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:53.364927 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:53.692810 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:53.692731 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:59.289857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.289817 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:59.391176 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.391132 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:59.531351 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.531318 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:53:59.531597 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.531575 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" containerID="cri-o://999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597" gracePeriod=30 Jul 08 11:53:59.567710 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.567572 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:53:59.969842 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:53:59.969791 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:00.400006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:00.399895 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:00.504573 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:00.504509 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:00.618255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:00.618206 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:00.845950 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:00.845906 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:04.792991 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.792922 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:54:04.796729 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796709 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796800 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796754 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796841 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796802 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796886 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796858 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796928 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796886 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcfkm\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796972 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796929 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.796972 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796951 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.797101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.796999 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.797101 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797029 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data\") pod \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\" (UID: \"89c046e5-6b68-4ace-ba4f-c9d5680daf1a\") " Jul 08 11:54:04.797256 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797229 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:04.797580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797368 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.797580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797393 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:04.797580 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797518 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:54:04.797918 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.797822 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:04.798219 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.798193 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:04.799106 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.799087 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:54:04.799275 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.799252 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:54:04.799677 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.799651 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:04.799784 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.799688 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm" (OuterVolumeSpecName: "kube-api-access-mcfkm") pod "89c046e5-6b68-4ace-ba4f-c9d5680daf1a" (UID: "89c046e5-6b68-4ace-ba4f-c9d5680daf1a"). InnerVolumeSpecName "kube-api-access-mcfkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:54:04.897809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897776 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.897809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897804 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.897809 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897813 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcfkm\" (UniqueName: \"kubernetes.io/projected/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-kube-api-access-mcfkm\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.898049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897823 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.898049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897831 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.898049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897839 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.898049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897846 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:04.898049 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:04.897853 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89c046e5-6b68-4ace-ba4f-c9d5680daf1a-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:05.179817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.179707 2559 generic.go:356] "Generic (PLEG): container finished" podID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerID="999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597" exitCode=0 Jul 08 11:54:05.179817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.179754 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" Jul 08 11:54:05.179817 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.179776 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerDied","Data":"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597"} Jul 08 11:54:05.180050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.179821 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr" event={"ID":"89c046e5-6b68-4ace-ba4f-c9d5680daf1a","Type":"ContainerDied","Data":"24c3fd0aecb38684679779fdb7034ed98d6895fcbfd471326e87adb5bdef7956"} Jul 08 11:54:05.180050 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.179847 2559 scope.go:117] "RemoveContainer" containerID="999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597" Jul 08 11:54:05.190016 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.189994 2559 scope.go:117] "RemoveContainer" containerID="50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286" Jul 08 11:54:05.198704 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.198683 2559 scope.go:117] "RemoveContainer" containerID="999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597" Jul 08 11:54:05.198972 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:54:05.198954 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597\": container with ID starting with 999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597 not found: ID does not exist" containerID="999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597" Jul 08 11:54:05.199019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.198981 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597"} err="failed to get container status \"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597\": rpc error: code = NotFound desc = could not find container \"999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597\": container with ID starting with 999c7864b3f0b535d04917cc798ab6d68f6e50a5f88f3b348f4b32d7d376e597 not found: ID does not exist" Jul 08 11:54:05.199019 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.199000 2559 scope.go:117] "RemoveContainer" containerID="50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286" Jul 08 11:54:05.199220 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:54:05.199203 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286\": container with ID starting with 50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286 not found: ID does not exist" containerID="50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286" Jul 08 11:54:05.199264 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.199224 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286"} err="failed to get container status \"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286\": rpc error: code = NotFound desc = could not find container \"50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286\": container with ID starting with 50415564c55d9fb86a9b78fa2e0adc8ad3d6a5d72e02e440a831a72d45331286 not found: ID does not exist" Jul 08 11:54:05.212331 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.212299 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:54:05.229994 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:05.229953 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-5a670c-openshift-default-6cd994659-prnvr"] Jul 08 11:54:06.170063 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.170030 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" path="/var/lib/kubelet/pods/89c046e5-6b68-4ace-ba4f-c9d5680daf1a/volumes" Jul 08 11:54:06.613062 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.613029 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:06.690214 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.690174 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:06.872138 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.872052 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:54:06.872314 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.872293 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" containerID="cri-o://8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916" gracePeriod=30 Jul 08 11:54:06.899889 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:06.899862 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:12.132828 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.132804 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:54:12.163860 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163779 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.163860 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163818 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163865 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163889 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163905 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.163984 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164076 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164020 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164350 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164104 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twnmf\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164350 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164161 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert\") pod \"1a10f2cd-df92-4234-a11e-80b998b93c5b\" (UID: \"1a10f2cd-df92-4234-a11e-80b998b93c5b\") " Jul 08 11:54:12.164350 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164221 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:12.164512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164420 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:12.164512 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164470 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.164621 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164610 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:12.164764 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.164739 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 11:54:12.165124 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.165093 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:12.166384 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.166357 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 11:54:12.166818 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.166793 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:54:12.166972 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.166949 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 11:54:12.167044 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.166996 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf" (OuterVolumeSpecName: "kube-api-access-twnmf") pod "1a10f2cd-df92-4234-a11e-80b998b93c5b" (UID: "1a10f2cd-df92-4234-a11e-80b998b93c5b"). InnerVolumeSpecName "kube-api-access-twnmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 11:54:12.228045 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.228010 2559 generic.go:356] "Generic (PLEG): container finished" podID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerID="8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916" exitCode=0 Jul 08 11:54:12.228233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.228047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerDied","Data":"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916"} Jul 08 11:54:12.228233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.228066 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" Jul 08 11:54:12.228233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.228118 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f" event={"ID":"1a10f2cd-df92-4234-a11e-80b998b93c5b","Type":"ContainerDied","Data":"635ed0ccc60558c22e8b5ec6380c068c128b2130220f476797a1a4450e8e38ee"} Jul 08 11:54:12.228233 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.228150 2559 scope.go:117] "RemoveContainer" containerID="8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916" Jul 08 11:54:12.237683 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.237664 2559 scope.go:117] "RemoveContainer" containerID="14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5" Jul 08 11:54:12.246190 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.246175 2559 scope.go:117] "RemoveContainer" containerID="8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916" Jul 08 11:54:12.246461 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:54:12.246437 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916\": container with ID starting with 8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916 not found: ID does not exist" containerID="8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916" Jul 08 11:54:12.246511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.246471 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916"} err="failed to get container status \"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916\": rpc error: code = NotFound desc = could not find container \"8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916\": container with ID starting with 8b7e2f6b3aad8e82c14c689be3c3da58eb8e4d9a56b9a5540d44bfe31d9e9916 not found: ID does not exist" Jul 08 11:54:12.246511 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.246488 2559 scope.go:117] "RemoveContainer" containerID="14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5" Jul 08 11:54:12.246819 ip-10-0-140-148 kubenswrapper[2559]: E0708 11:54:12.246795 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5\": container with ID starting with 14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5 not found: ID does not exist" containerID="14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5" Jul 08 11:54:12.246915 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.246835 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5"} err="failed to get container status \"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5\": rpc error: code = NotFound desc = could not find container \"14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5\": container with ID starting with 14a21d9d0fb4355f8c24c4b7d2c787b33790b75c39fe9ecbae60e6f9adf2deb5 not found: ID does not exist" Jul 08 11:54:12.251480 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.251452 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:54:12.256386 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.256361 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e79b1c-openshift-default-b66798687-xsr9f"] Jul 08 11:54:12.265596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265574 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-envoy\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265596 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265597 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-token\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265611 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-podinfo\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265623 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-credential-socket\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265663 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-istio-data\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265676 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a10f2cd-df92-4234-a11e-80b998b93c5b-workload-certs\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265687 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twnmf\" (UniqueName: \"kubernetes.io/projected/1a10f2cd-df92-4234-a11e-80b998b93c5b-kube-api-access-twnmf\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.265743 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.265698 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a10f2cd-df92-4234-a11e-80b998b93c5b-istiod-ca-cert\") on node \"ip-10-0-140-148.ec2.internal\" DevicePath \"\"" Jul 08 11:54:12.357303 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.357259 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lngzl_f996741e-93d9-49a1-8003-79ca3775ba11/manager/0.log" Jul 08 11:54:12.487774 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.487743 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zfg2n_34a7f79c-7d3d-4e08-8db2-78e705308a85/manager/0.log" Jul 08 11:54:12.629923 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:12.629894 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b9657c48d-mz6lj_18d25f9e-f54c-46da-9666-0ec88e48327f/manager/0.log" Jul 08 11:54:13.200232 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:13.200201 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9b86684-ltxbf_5fe216ce-fb37-48af-84b5-5124b2322006/copy-manifests/0.log" Jul 08 11:54:13.209131 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:13.209105 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9b86684-ltxbf_5fe216ce-fb37-48af-84b5-5124b2322006/manager/0.log" Jul 08 11:54:14.143165 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.143135 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/util/0.log" Jul 08 11:54:14.154874 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.154846 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/pull/0.log" Jul 08 11:54:14.162776 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.162748 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/extract/0.log" Jul 08 11:54:14.169449 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.169413 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" path="/var/lib/kubelet/pods/1a10f2cd-df92-4234-a11e-80b998b93c5b/volumes" Jul 08 11:54:14.288485 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.288450 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/util/0.log" Jul 08 11:54:14.300534 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.300510 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/pull/0.log" Jul 08 11:54:14.309000 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.308976 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/extract/0.log" Jul 08 11:54:14.429723 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.429626 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/util/0.log" Jul 08 11:54:14.437434 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.437409 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/pull/0.log" Jul 08 11:54:14.450162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.450138 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/extract/0.log" Jul 08 11:54:14.567257 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.567232 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/util/0.log" Jul 08 11:54:14.576041 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.576016 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/pull/0.log" Jul 08 11:54:14.584649 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.584611 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/extract/0.log" Jul 08 11:54:14.967183 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:14.967152 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7nl7d_3104da5f-fe19-4ea5-ace1-f1930cfc54ea/manager/0.log" Jul 08 11:54:15.342544 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:15.342452 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pvrnh_af880455-2db5-4b1e-83c7-ffd3e24e2e5b/manager/0.log" Jul 08 11:54:15.937986 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:15.937956 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dxcq4_7067d7ad-0ca3-4806-a452-f528d6ac09d9/istio-proxy/0.log" Jul 08 11:54:16.416336 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:16.416302 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/1.log" Jul 08 11:54:16.537149 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:16.537116 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-pph62_3a006a2d-051a-47a9-a46f-80fe737267ad/payload-pre-processing/0.log" Jul 08 11:54:16.666954 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:16.666876 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-v5kzn_aaaa7957-7312-4181-8fd1-c3d6e8db4f19/payload-processing/0.log" Jul 08 11:54:16.790791 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:16.790762 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d96df899f-2wt7h_a02b54ca-d226-4494-bc5f-c3a28d197408/router/0.log" Jul 08 11:54:17.262702 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:17.262677 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pf7sr_ab6a3133-9ed8-4053-8786-20abdef4e66e/main/0.log" Jul 08 11:54:17.593761 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:17.593672 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-c64vz_c22f0dc8-83d5-48e2-90bb-7fa2c2fff7d3/main/0.log" Jul 08 11:54:21.527176 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:21.527143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:21.676274 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:21.676237 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" limit=1024 Jul 08 11:54:24.902191 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:24.902153 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w5f7p_b358f565-efb3-4820-b6df-d5aed770cbb2/global-pull-secret-syncer/0.log" Jul 08 11:54:25.001677 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:25.001624 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6bctf_8132bc83-8fd5-45e4-a58c-947741a8119f/konnectivity-agent/0.log" Jul 08 11:54:25.224120 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:25.224092 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-148.ec2.internal_2339c7fd5cf13a79b0441cd2cf2b5de7/haproxy/0.log" Jul 08 11:54:29.350433 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.350401 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/extract/0.log" Jul 08 11:54:29.386574 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.386538 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/util/0.log" Jul 08 11:54:29.413857 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.413821 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mkq8_5d5d5ca0-f052-42ca-a0dc-9571c0eeda2e/pull/0.log" Jul 08 11:54:29.463339 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.463313 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/extract/0.log" Jul 08 11:54:29.491595 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.491561 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/util/0.log" Jul 08 11:54:29.517427 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.517395 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cq7n4_4cef19d6-a3b6-44b8-867a-b29147720472/pull/0.log" Jul 08 11:54:29.548020 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.547983 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/extract/0.log" Jul 08 11:54:29.574812 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.574780 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/util/0.log" Jul 08 11:54:29.603465 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.603376 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f2kwp_8fc9a805-a906-471c-9573-9355c0fd9263/pull/0.log" Jul 08 11:54:29.632473 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.632440 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/extract/0.log" Jul 08 11:54:29.660927 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.660900 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/util/0.log" Jul 08 11:54:29.695978 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:29.695949 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1n2jvn_88a62727-1d90-4b83-b75b-ff5a98957127/pull/0.log" Jul 08 11:54:30.169377 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:30.169332 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7nl7d_3104da5f-fe19-4ea5-ace1-f1930cfc54ea/manager/0.log" Jul 08 11:54:30.393691 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:30.393657 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pvrnh_af880455-2db5-4b1e-83c7-ffd3e24e2e5b/manager/0.log" Jul 08 11:54:32.396197 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:32.396160 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-9l67d_7c62768a-f097-4ddb-b07b-835fd693b3c5/cluster-monitoring-operator/0.log" Jul 08 11:54:32.664262 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:32.664177 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5kcf5_234e8778-b528-4487-90ca-1d3de9d6ab10/node-exporter/0.log" Jul 08 11:54:32.696189 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:32.696156 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5kcf5_234e8778-b528-4487-90ca-1d3de9d6ab10/kube-rbac-proxy/0.log" Jul 08 11:54:32.728021 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:32.727998 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5kcf5_234e8778-b528-4487-90ca-1d3de9d6ab10/init-textfile/0.log" Jul 08 11:54:34.395617 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.395585 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv"] Jul 08 11:54:34.396006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.395982 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" Jul 08 11:54:34.396006 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.395995 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" Jul 08 11:54:34.396077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396016 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396022 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396030 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396077 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396036 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396093 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396102 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a10f2cd-df92-4234-a11e-80b998b93c5b" containerName="istio-proxy" Jul 08 11:54:34.396204 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.396108 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c046e5-6b68-4ace-ba4f-c9d5680daf1a" containerName="istio-proxy" Jul 08 11:54:34.400909 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.400882 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.403771 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.403748 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wcxfn\"/\"openshift-service-ca.crt\"" Jul 08 11:54:34.404607 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.404585 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wcxfn\"/\"default-dockercfg-kjnzf\"" Jul 08 11:54:34.404821 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.404585 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wcxfn\"/\"kube-root-ca.crt\"" Jul 08 11:54:34.408792 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.408770 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv"] Jul 08 11:54:34.479142 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.479106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-proc\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.479324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.479178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wktxn\" (UniqueName: \"kubernetes.io/projected/89fa0b1c-678c-40e6-963c-9ee3cf54399c-kube-api-access-wktxn\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.479324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.479218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-sys\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.479324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.479242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-podres\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.479324 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.479285 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-lib-modules\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579751 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-proc\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579777 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wktxn\" (UniqueName: \"kubernetes.io/projected/89fa0b1c-678c-40e6-963c-9ee3cf54399c-kube-api-access-wktxn\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-sys\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-podres\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579845 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-proc\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-lib-modules\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-sys\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.579969 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579972 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-lib-modules\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.580286 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.579973 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fa0b1c-678c-40e6-963c-9ee3cf54399c-podres\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.589585 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.589561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wktxn\" (UniqueName: \"kubernetes.io/projected/89fa0b1c-678c-40e6-963c-9ee3cf54399c-kube-api-access-wktxn\") pod \"perf-node-gather-daemonset-zw2mv\" (UID: \"89fa0b1c-678c-40e6-963c-9ee3cf54399c\") " pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.707740 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.707696 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-m7sbh_d96103c4-bf70-4902-9d90-d70e6c6b2a9c/networking-console-plugin/0.log" Jul 08 11:54:34.712599 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.712575 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:34.860370 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:34.860342 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv"] Jul 08 11:54:34.862016 ip-10-0-140-148 kubenswrapper[2559]: W0708 11:54:34.861986 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod89fa0b1c_678c_40e6_963c_9ee3cf54399c.slice/crio-0bf6bb26d6783af68c523961104b489d8caf5d2b46f6b4942dcc58beaadf1a6f WatchSource:0}: Error finding container 0bf6bb26d6783af68c523961104b489d8caf5d2b46f6b4942dcc58beaadf1a6f: Status 404 returned error can't find the container with id 0bf6bb26d6783af68c523961104b489d8caf5d2b46f6b4942dcc58beaadf1a6f Jul 08 11:54:35.354871 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.354835 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" event={"ID":"89fa0b1c-678c-40e6-963c-9ee3cf54399c","Type":"ContainerStarted","Data":"2e0a7a71187274f490521c93c8120d4e1070530a356b4f0da06ac369370981ae"} Jul 08 11:54:35.355036 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.354897 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:35.355036 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.354915 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" event={"ID":"89fa0b1c-678c-40e6-963c-9ee3cf54399c","Type":"ContainerStarted","Data":"0bf6bb26d6783af68c523961104b489d8caf5d2b46f6b4942dcc58beaadf1a6f"} Jul 08 11:54:35.364516 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.364491 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/1.log" Jul 08 11:54:35.378162 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.378141 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-zpsrs_a1e48b41-a320-49ad-89ed-6cd72d6f1bcb/console-operator/2.log" Jul 08 11:54:35.382540 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:35.382500 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" podStartSLOduration=1.382486305 podStartE2EDuration="1.382486305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 11:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 11:54:35.382456469 +0000 UTC m=+2491.764103240" watchObservedRunningTime="2026-07-08 11:54:35.382486305 +0000 UTC m=+2491.764133076" Jul 08 11:54:36.565192 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:36.565154 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-8vjds_056b73f6-ff1d-49f9-b96b-d17a2c513b0e/volume-data-source-validator/0.log" Jul 08 11:54:37.710749 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:37.710721 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lkn52_d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37/dns/0.log" Jul 08 11:54:37.758002 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:37.757971 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lkn52_d48f25bc-6a76-49eb-b1ae-6fd38a8c0a37/kube-rbac-proxy/0.log" Jul 08 11:54:37.887137 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:37.887107 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2hf4f_ac3d9ce1-d290-4f67-80d2-98e05b218cc2/dns-node-resolver/0.log" Jul 08 11:54:38.625657 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:38.625611 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-blkb4_320a10ed-50a0-4a8e-8246-167e3a923848/node-ca/0.log" Jul 08 11:54:39.628936 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:39.628902 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dxcq4_7067d7ad-0ca3-4806-a452-f528d6ac09d9/istio-proxy/0.log" Jul 08 11:54:39.946469 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:39.946380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/0.log" Jul 08 11:54:39.999134 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:39.999094 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/1.log" Jul 08 11:54:40.069498 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:40.069464 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-pph62_3a006a2d-051a-47a9-a46f-80fe737267ad/payload-pre-processing/0.log" Jul 08 11:54:40.155115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:40.155082 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-v5kzn_aaaa7957-7312-4181-8fd1-c3d6e8db4f19/payload-processing/0.log" Jul 08 11:54:40.205725 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:40.205702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d96df899f-2wt7h_a02b54ca-d226-4494-bc5f-c3a28d197408/router/0.log" Jul 08 11:54:40.948549 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:40.948517 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9hns5_fbc898d5-8f90-406f-a2d0-bae27b06dff4/serve-healthcheck-canary/0.log" Jul 08 11:54:41.372316 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.372279 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wcxfn/perf-node-gather-daemonset-zw2mv" Jul 08 11:54:41.563513 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.563476 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2zrxz_37ba2488-948f-43c4-b05e-332f112544b7/insights-operator/0.log" Jul 08 11:54:41.566255 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.566231 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2zrxz_37ba2488-948f-43c4-b05e-332f112544b7/insights-operator/1.log" Jul 08 11:54:41.686199 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.686108 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qcs6_95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2/kube-rbac-proxy/0.log" Jul 08 11:54:41.711833 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.711802 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qcs6_95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2/exporter/0.log" Jul 08 11:54:41.737202 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:41.737170 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7qcs6_95f9c2cf-9e8c-4601-aa7d-f3fa26e5e5e2/extractor/0.log" Jul 08 11:54:44.163496 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.163403 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lngzl_f996741e-93d9-49a1-8003-79ca3775ba11/manager/0.log" Jul 08 11:54:44.346115 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.346083 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zfg2n_34a7f79c-7d3d-4e08-8db2-78e705308a85/manager/0.log" Jul 08 11:54:44.481618 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.481577 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b9657c48d-mz6lj_18d25f9e-f54c-46da-9666-0ec88e48327f/manager/0.log" Jul 08 11:54:44.553254 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.553222 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dxcq4_7067d7ad-0ca3-4806-a452-f528d6ac09d9/istio-proxy/0.log" Jul 08 11:54:44.796728 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.796619 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9b86684-ltxbf_5fe216ce-fb37-48af-84b5-5124b2322006/manager/0.log" Jul 08 11:54:44.806093 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.806065 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/0.log" Jul 08 11:54:44.823421 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.823362 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79c9b86684-ltxbf_5fe216ce-fb37-48af-84b5-5124b2322006/copy-manifests/0.log" Jul 08 11:54:44.864713 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.864674 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/1.log" Jul 08 11:54:44.907284 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.907249 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-pph62_3a006a2d-051a-47a9-a46f-80fe737267ad/payload-pre-processing/0.log" Jul 08 11:54:44.968023 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.967994 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-v5kzn_aaaa7957-7312-4181-8fd1-c3d6e8db4f19/payload-processing/0.log" Jul 08 11:54:44.992917 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:44.992887 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d96df899f-2wt7h_a02b54ca-d226-4494-bc5f-c3a28d197408/router/0.log" Jul 08 11:54:46.106623 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.106595 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-574bb547d7-h4ch5_aa0b7cec-8c6a-4284-90b0-320d273fe156/manager/0.log" Jul 08 11:54:46.139281 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.139246 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4dxcq4_7067d7ad-0ca3-4806-a452-f528d6ac09d9/istio-proxy/0.log" Jul 08 11:54:46.347128 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.347098 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/0.log" Jul 08 11:54:46.403784 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.403752 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vdk7x_162da7b4-b5e9-4a65-aaf0-e98d4dc6ed33/istio-proxy/1.log" Jul 08 11:54:46.458221 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.458192 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-pph62_3a006a2d-051a-47a9-a46f-80fe737267ad/payload-pre-processing/0.log" Jul 08 11:54:46.517835 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.517801 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-v5kzn_aaaa7957-7312-4181-8fd1-c3d6e8db4f19/payload-processing/0.log" Jul 08 11:54:46.539977 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:46.539950 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d96df899f-2wt7h_a02b54ca-d226-4494-bc5f-c3a28d197408/router/0.log" Jul 08 11:54:50.804962 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:50.804930 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vfkb8_61dfaeb2-3703-42fd-a32a-cb2f495a564f/migrator/0.log" Jul 08 11:54:50.832837 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:50.832807 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-vfkb8_61dfaeb2-3703-42fd-a32a-cb2f495a564f/graceful-termination/0.log" Jul 08 11:54:51.289418 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:51.289389 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-85mhl_0393330b-f585-4071-afa5-7740af485277/kube-storage-version-migrator-operator/1.log" Jul 08 11:54:51.291015 ip-10-0-140-148 kubenswrapper[2559]: I0708 11:54:51.290989 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-85mhl_0393330b-f585-4071-afa5-7740af485277/kube-storage-version-migrator-operator/0.log"