Jul 12 15:20:13.205602 ip-10-0-142-230 systemd[1]: Starting Kubernetes Kubelet... Jul 12 15:20:13.556144 ip-10-0-142-230 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:13.556144 ip-10-0-142-230 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 15:20:13.556144 ip-10-0-142-230 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:13.556144 ip-10-0-142-230 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 15:20:13.556144 ip-10-0-142-230 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:13.557718 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.557625 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 15:20:13.561588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561571 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.561588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561588 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561592 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561595 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561600 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561605 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561608 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561612 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561615 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561617 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561620 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561623 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561626 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561634 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561637 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561640 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561642 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561645 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561647 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561650 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.561656 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561652 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561655 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561657 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561660 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561663 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561666 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561669 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561672 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561674 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561677 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561679 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561682 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561684 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561686 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561689 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561691 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561693 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561696 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561699 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561701 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.562098 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561704 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561706 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561708 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561712 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561714 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561716 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561720 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561723 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561726 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561728 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561731 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561733 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561736 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561740 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561742 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561745 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561748 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561750 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561753 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.562588 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561755 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561758 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561761 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561764 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561766 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561770 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561772 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561775 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561777 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561780 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561784 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561786 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561789 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561791 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561794 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561796 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561798 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561802 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561804 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561807 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.563087 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561809 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561811 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561814 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561816 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561819 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561823 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561827 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561830 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561833 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561836 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561839 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561842 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561845 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561847 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561850 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561852 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561855 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561857 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561860 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.561862 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.563846 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562587 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562594 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562597 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562601 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562604 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562607 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562609 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562612 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562615 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562617 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562621 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562623 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562626 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562628 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562631 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562633 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562636 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562639 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562642 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562644 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.564409 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562647 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562650 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562652 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562654 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562657 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562659 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562662 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562664 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562667 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562669 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562672 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562674 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562677 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562679 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562682 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562684 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562687 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562690 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562692 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.564892 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562697 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562700 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562703 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562706 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562709 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562712 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562715 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562717 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562720 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562723 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562726 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562730 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562734 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562737 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562740 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562742 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562745 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562748 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562750 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562753 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.565367 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562756 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562759 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562762 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562764 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562767 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562770 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562772 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562776 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562780 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562782 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562785 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562787 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562791 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562793 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562796 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562798 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562801 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562804 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562806 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.565848 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562809 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562812 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562814 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562817 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562820 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562822 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562825 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562827 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562830 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562832 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562835 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562837 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562840 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562842 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562845 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562847 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562850 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562852 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562855 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562857 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.566333 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.562860 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564484 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564624 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564636 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564641 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564646 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564649 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564653 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564656 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564659 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564662 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564665 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564668 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564671 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564673 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564676 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564678 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564681 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564684 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564686 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.566938 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564689 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564692 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564695 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564697 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564700 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564704 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564706 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564709 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564712 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564714 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564717 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564720 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564722 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564725 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564729 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564732 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564735 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564742 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564746 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.567437 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564749 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564751 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564754 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564757 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564759 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564762 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564764 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564767 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564769 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564772 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564774 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564777 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564779 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564782 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564785 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564788 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564790 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564793 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564796 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564798 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.567888 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564801 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564804 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564806 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564809 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564811 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564814 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564818 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564821 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564824 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564827 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564829 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564832 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564836 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564839 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564843 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564846 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564849 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564853 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564856 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.568439 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564859 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564861 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564864 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564867 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564869 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564872 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564874 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564877 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564880 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564882 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564885 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564887 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564890 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564892 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564895 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564898 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564901 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564903 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564906 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564909 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.568901 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564912 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.564915 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.564994 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565003 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565010 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565015 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565019 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565023 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565027 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565031 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565034 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565037 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565040 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565044 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565047 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565050 2563 flags.go:64] FLAG: --cgroup-root="" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565053 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565056 2563 flags.go:64] FLAG: --client-ca-file="" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565059 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565063 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565068 2563 flags.go:64] FLAG: --cluster-domain="" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565070 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565073 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565076 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 15:20:13.569404 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565080 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565083 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565086 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565089 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565092 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565095 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565098 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565100 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565104 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565107 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565112 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565115 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565118 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565121 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565124 2563 flags.go:64] FLAG: --enable-server="true" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565126 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565134 2563 flags.go:64] FLAG: --event-burst="100" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565138 2563 flags.go:64] FLAG: --event-qps="50" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565141 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565144 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565147 2563 flags.go:64] FLAG: --eviction-hard="" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565150 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565153 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565156 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565159 2563 flags.go:64] FLAG: --eviction-soft="" Jul 12 15:20:13.569955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565162 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565165 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565168 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565172 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565175 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565178 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565181 2563 flags.go:64] FLAG: --feature-gates="" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565185 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565188 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565191 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565194 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565197 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565200 2563 flags.go:64] FLAG: --help="false" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565203 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565206 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565210 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565213 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565216 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565219 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565222 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565225 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565228 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565231 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 15:20:13.570618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565234 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565237 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565240 2563 flags.go:64] FLAG: --kube-reserved="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565243 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565245 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565248 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565251 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565254 2563 flags.go:64] FLAG: --lock-file="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565257 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565260 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565263 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565268 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565271 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565275 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565278 2563 flags.go:64] FLAG: --logging-format="text" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565281 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565284 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565286 2563 flags.go:64] FLAG: --manifest-url="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565289 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565294 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565297 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565301 2563 flags.go:64] FLAG: --max-pods="110" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565304 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565308 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565311 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 15:20:13.571194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565314 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565332 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565335 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565338 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565346 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565349 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565352 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565355 2563 flags.go:64] FLAG: --pod-cidr="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565358 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565363 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565366 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565369 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565372 2563 flags.go:64] FLAG: --port="10250" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565376 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565379 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03f5c6df706c03062" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565382 2563 flags.go:64] FLAG: --qos-reserved="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565385 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565388 2563 flags.go:64] FLAG: --register-node="true" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565391 2563 flags.go:64] FLAG: --register-with-taints="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565395 2563 flags.go:64] FLAG: --registry-burst="10" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565398 2563 flags.go:64] FLAG: --registry-qps="5" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565401 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565404 2563 flags.go:64] FLAG: --reserved-memory="" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565407 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565410 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 15:20:13.571893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565413 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565416 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565419 2563 flags.go:64] FLAG: --runonce="false" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565422 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565425 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565431 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565434 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565437 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565440 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565443 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565446 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565449 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565452 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565455 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565458 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565462 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565465 2563 flags.go:64] FLAG: --system-cgroups="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565467 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565473 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565475 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565478 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565482 2563 flags.go:64] FLAG: --tls-min-version="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565485 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565487 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565490 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 15:20:13.572515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565493 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565496 2563 flags.go:64] FLAG: --v="2" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565501 2563 flags.go:64] FLAG: --version="false" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565505 2563 flags.go:64] FLAG: --vmodule="" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565509 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565512 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565630 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565634 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565636 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565639 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565642 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565645 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565649 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565652 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565655 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565657 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565660 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565663 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565665 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565668 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565670 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.573116 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565673 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565675 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565678 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565680 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565683 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565685 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565687 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565690 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565694 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565698 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565702 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565705 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565707 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565710 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565713 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565716 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565719 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565721 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565723 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.573650 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565726 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565729 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565731 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565733 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565737 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565740 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565742 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565744 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565747 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565749 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565752 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565754 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565757 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565759 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565762 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565764 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565767 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565769 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565773 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.574112 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565776 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565779 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565783 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565786 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565788 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565791 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565793 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565796 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565798 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565801 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565804 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565807 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565809 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565812 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565814 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565817 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565820 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565826 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565828 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565831 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.574673 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565834 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565836 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565839 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565843 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565847 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565849 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565852 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565855 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565859 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565862 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565864 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565867 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565869 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565872 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565875 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565878 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565880 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565883 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565885 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565888 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.575146 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565890 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565893 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565895 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565898 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565901 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.565903 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.565913 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.572856 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.572875 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.573612 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.573620 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573682 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573687 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573691 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573694 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.575652 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573697 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573700 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573703 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573706 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573709 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573712 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573715 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573718 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573720 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573723 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573726 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573728 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573731 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573734 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573737 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573739 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573742 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573745 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573747 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573750 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.576018 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573752 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573757 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573760 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573763 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573765 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573769 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573772 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573774 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573777 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573779 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573782 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573785 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573788 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573791 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573793 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573796 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573798 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573800 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573803 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.576506 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573806 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573808 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573811 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573814 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573816 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573819 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573821 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573823 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573826 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573828 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573832 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573836 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573840 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573843 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573846 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573849 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573851 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573854 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573857 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.577009 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573860 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573862 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573867 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573870 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573873 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573876 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573878 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573881 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573883 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573886 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573888 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573891 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573893 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573896 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573898 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573901 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573903 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573906 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573908 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573911 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.577516 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573913 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573915 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573918 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573920 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573923 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573925 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573928 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573930 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573933 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573938 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573942 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573945 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573948 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573951 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573954 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573956 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.578014 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.573959 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.573964 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574079 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574083 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574086 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574088 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574091 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574094 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574097 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574099 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574102 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574104 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574107 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574109 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574111 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:13.578476 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574114 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574117 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574120 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574122 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574125 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574128 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574130 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574133 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574135 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574138 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574140 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574143 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574146 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574149 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574151 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574154 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574156 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574159 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574161 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574163 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:13.578843 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574166 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574168 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574171 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574173 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574175 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574178 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574180 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574183 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574185 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574188 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574191 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574193 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574196 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574198 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574201 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574203 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574205 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574208 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574213 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:13.579346 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574217 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574220 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574223 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574225 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574228 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574231 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574234 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574237 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574240 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574242 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574245 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574247 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574250 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574252 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574254 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574257 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574260 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574262 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574265 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574267 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:13.579813 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574269 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574272 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574274 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574284 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574288 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574290 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574293 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574296 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574298 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574301 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574303 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574305 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574308 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574312 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574332 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574336 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574339 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574345 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574350 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:13.580281 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574353 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574356 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574358 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574362 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574364 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574367 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574370 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:13.574372 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.574377 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.574507 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.576493 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.577719 2563 server.go:1019] "Starting client certificate rotation" Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.577818 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 15:20:13.580795 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.577858 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 15:20:13.581124 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.579863 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 15:20:13.581124 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.581063 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 15:20:13.589952 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.589934 2563 log.go:25] "Validated CRI v1 runtime API" Jul 12 15:20:13.593948 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.593930 2563 log.go:25] "Validated CRI v1 image API" Jul 12 15:20:13.596965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.596945 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 15:20:13.599823 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.599792 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9d18bcba-dea4-47b6-aaea-8b2f28cf9ebf:/dev/nvme0n1p4 acb09efa-badb-4e35-8fa0-83f48b782d1d:/dev/nvme0n1p3] Jul 12 15:20:13.599926 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.599824 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 15:20:13.605084 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.605059 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 15:20:13.605177 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.605005 2563 manager.go:216] Machine: {Timestamp:2026-07-12 15:20:13.604439433 +0000 UTC m=+0.297718428 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099407 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24994d4d5cdf58ae7abe189973455c SystemUUID:ec24994d-4d5c-df58-ae7a-be189973455c BootID:5c6f0a40-6427-4c8d-a462-eae66865ae12 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:37:df:62:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:37:df:62:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:62:d6:fd:3a:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 15:20:13.605177 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.605173 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 15:20:13.605345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.605293 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 15:20:13.608541 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.608512 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 15:20:13.608684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.608544 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 15:20:13.609160 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.609151 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 15:20:13.609193 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.609164 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 15:20:13.609193 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.609171 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 15:20:13.609869 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.609859 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 15:20:13.610573 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.610563 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 15:20:13.613722 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613710 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 12 15:20:13.613847 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613837 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 15:20:13.613876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613869 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 12 15:20:13.613912 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613878 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 15:20:13.613912 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613892 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 15:20:13.613912 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613901 2563 kubelet.go:398] "Adding apiserver pod source" Jul 12 15:20:13.613912 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.613909 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 15:20:13.614380 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.614366 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 15:20:13.614429 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.614388 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 15:20:13.616204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.616186 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 15:20:13.617457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617444 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 15:20:13.617512 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617475 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 15:20:13.617640 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617630 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 15:20:13.617684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617647 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 15:20:13.617684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617653 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 15:20:13.617684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617659 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 15:20:13.617684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617667 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 15:20:13.617684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.617673 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 15:20:13.618066 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618058 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 15:20:13.618066 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618067 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 15:20:13.618122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618074 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 15:20:13.618122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618080 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 15:20:13.618122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618089 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 15:20:13.618122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.618096 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 15:20:13.619367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.619354 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 15:20:13.619400 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.619369 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 15:20:13.622798 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.622783 2563 server.go:1268] "Started kubelet" Jul 12 15:20:13.622889 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.622860 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 15:20:13.623490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.623439 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 15:20:13.623595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.623512 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 15:20:13.623827 ip-10-0-142-230 systemd[1]: Started Kubernetes Kubelet. Jul 12 15:20:13.624260 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.624238 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 15:20:13.624861 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.624701 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 15:20:13.624910 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.624846 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 15:20:13.624910 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.624701 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 15:20:13.625904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.625889 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 12 15:20:13.630738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.627590 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-230.ec2.internal.18c193f65598f478 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-230.ec2.internal,UID:ip-10-0-142-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-230.ec2.internal,},FirstTimestamp:2026-07-12 15:20:13.622752376 +0000 UTC m=+0.316031371,LastTimestamp:2026-07-12 15:20:13.622752376 +0000 UTC m=+0.316031371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-230.ec2.internal,}" Jul 12 15:20:13.631024 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.631004 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 15:20:13.634435 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.634415 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 15:20:13.634524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.634417 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 15:20:13.634646 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.634630 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:13.634921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.634904 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 15:20:13.635055 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635042 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 15:20:13.635115 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.634940 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 15:20:13.635176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635141 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 12 15:20:13.635176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635151 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 12 15:20:13.635558 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635542 2563 factory.go:55] Registering systemd factory Jul 12 15:20:13.635637 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635562 2563 factory.go:223] Registration of the systemd container factory successfully Jul 12 15:20:13.635818 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635794 2563 factory.go:153] Registering CRI-O factory Jul 12 15:20:13.635818 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635813 2563 factory.go:223] Registration of the crio container factory successfully Jul 12 15:20:13.635930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635864 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 15:20:13.635930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635892 2563 factory.go:103] Registering Raw factory Jul 12 15:20:13.635930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.635906 2563 manager.go:1196] Started watching for new ooms in manager Jul 12 15:20:13.636309 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.636296 2563 manager.go:349] Starting recovery of all containers Jul 12 15:20:13.644556 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.644422 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 15:20:13.646144 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.646091 2563 manager.go:354] Recovery completed Jul 12 15:20:13.647196 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.647168 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 15:20:13.647303 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.647280 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 15:20:13.651081 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.651067 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.653132 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653109 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.653185 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653145 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.653185 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653156 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.653474 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653458 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m9h5k" Jul 12 15:20:13.653707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653691 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 15:20:13.653707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653700 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.653707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653706 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 15:20:13.653857 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.653726 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 12 15:20:13.654410 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654392 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.654477 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654423 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.654477 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654437 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.654906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654894 2563 policy_none.go:49] "None policy: Start" Jul 12 15:20:13.654941 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654910 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 15:20:13.654941 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.654921 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 15:20:13.655742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.655732 2563 policy_none.go:47] "Start" Jul 12 15:20:13.661147 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.661130 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m9h5k" Jul 12 15:20:13.663844 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.663776 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-230.ec2.internal.18c193f6576881e2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-230.ec2.internal,UID:ip-10-0-142-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-230.ec2.internal,},FirstTimestamp:2026-07-12 15:20:13.653131746 +0000 UTC m=+0.346410740,LastTimestamp:2026-07-12 15:20:13.653131746 +0000 UTC m=+0.346410740,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-230.ec2.internal,}" Jul 12 15:20:13.698995 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.698975 2563 manager.go:337] "Starting Device Plugin manager" Jul 12 15:20:13.699105 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.699015 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 15:20:13.699105 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699025 2563 server.go:87] "Starting device plugin registration server" Jul 12 15:20:13.699306 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699294 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 15:20:13.699386 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699309 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 15:20:13.699440 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699393 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 15:20:13.699511 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699498 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 15:20:13.699555 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.699511 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 15:20:13.700088 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.700062 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 15:20:13.700184 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.700124 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:13.780311 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.780284 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 15:20:13.781289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.780336 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 15:20:13.781289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.780360 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 15:20:13.781289 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.780390 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 15:20:13.782827 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.782807 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:13.799923 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.799894 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.801042 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.801023 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.801138 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.801058 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.801138 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.801074 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.801138 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.801105 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.811377 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.811307 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.811377 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.811348 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-230.ec2.internal\": node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:13.834883 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.834849 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:13.880548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.880499 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"] Jul 12 15:20:13.880722 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.880600 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.881611 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.881594 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.881702 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.881626 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.881702 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.881641 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.883003 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.882991 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.883113 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.883098 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.883171 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.883130 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.883718 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.883702 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.883806 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.883733 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.883806 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.883748 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.884452 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.884435 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.884541 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.884464 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.884541 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.884474 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.884740 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.884724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.884821 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.884747 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:13.885363 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.885347 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:13.885445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.885378 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:13.885445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.885389 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:13.904917 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.904896 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.908850 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.908836 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-230.ec2.internal\" not found" node="ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.935543 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:13.935511 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:13.936612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.936594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.936668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.936621 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:13.936668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:13.936640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a3ddb42b509e449091e6f0bf0e70c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"a5a3ddb42b509e449091e6f0bf0e70c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.036229 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.036181 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.037345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a3ddb42b509e449091e6f0bf0e70c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"a5a3ddb42b509e449091e6f0bf0e70c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.037345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.037499 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037374 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.037499 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037411 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5a3ddb42b509e449091e6f0bf0e70c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-230.ec2.internal\" (UID: \"a5a3ddb42b509e449091e6f0bf0e70c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.037499 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.037499 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.037426 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a325171318a9024f14dff1bce3655e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal\" (UID: \"5a325171318a9024f14dff1bce3655e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.137038 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.136961 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.206259 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.206225 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.211877 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.211858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.237828 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.237799 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.338301 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.338254 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.438739 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.438651 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.464925 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.464901 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:14.539336 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.539284 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-230.ec2.internal\" not found" Jul 12 15:20:14.577537 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.577518 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 15:20:14.578082 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.577673 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 15:20:14.578082 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.577714 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 15:20:14.593029 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.593004 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:14.615018 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.614990 2563 apiserver.go:52] "Watching apiserver" Jul 12 15:20:14.623431 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.623408 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 15:20:14.623766 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.623745 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-b6qnh","openshift-multus/multus-75kzc","openshift-multus/network-metrics-daemon-dtq8d","openshift-network-diagnostics/network-check-target-l9fw9","openshift-network-operator/iptables-alerter-wtbr2","kube-system/global-pull-secret-syncer-h7cqj","kube-system/konnectivity-agent-42264","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7","openshift-multus/multus-additional-cni-plugins-gktwq","openshift-ovn-kubernetes/ovnkube-node-58qlj","openshift-cluster-node-tuning-operator/tuned-ggwwf","openshift-dns/node-resolver-77n27"] Jul 12 15:20:14.625113 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.625092 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.625182 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.625167 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:14.626174 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.626156 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:14.626256 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.626207 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:14.627355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.627342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.628365 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.628348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.630108 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.630081 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 15:20:14.630202 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.630132 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cfqkk\"" Jul 12 15:20:14.630409 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.630391 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.630409 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.630397 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.630577 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.630562 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.631030 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.631011 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.631030 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.631024 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.631156 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.631094 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w8qnk\"" Jul 12 15:20:14.631827 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.631354 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.632582 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.632560 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.632676 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.632627 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:14.632988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.632970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.633146 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633131 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ltxg5\"" Jul 12 15:20:14.633226 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633182 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 15:20:14.633463 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633443 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 15:20:14.633463 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633462 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.633592 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633562 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.633738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633716 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.633942 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 15:20:14.634020 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633942 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xsn9m\"" Jul 12 15:20:14.634020 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.633979 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 15:20:14.634150 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.634134 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.634628 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.634614 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 15:20:14.634690 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.634628 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.634831 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.634818 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.636181 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.636162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.637538 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.637521 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.637636 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.637572 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 15:20:14.637696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.637667 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rnvk9\"" Jul 12 15:20:14.637696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.637678 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 15:20:14.637926 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.637913 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9dvrq\"" Jul 12 15:20:14.638201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.638185 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 15:20:14.638289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.638268 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 15:20:14.638652 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.638632 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 15:20:14.638729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.638663 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.638922 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.638906 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.639028 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.639014 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 15:20:14.639073 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.639040 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4phfh\"" Jul 12 15:20:14.639073 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.639046 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 15:20:14.639143 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.639095 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 15:20:14.639365 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.639347 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:14.640612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-os-release\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.640612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-var-lib-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-node-log\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640617 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-system-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.640713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640664 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-systemd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-multus-certs\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-log-socket\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640745 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3c7a6a4-f419-42e8-be7a-f64e2d254290-agent-certs\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-socket-dir-parent\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640785 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-daemon-config\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640798 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n9jk\" (UniqueName: \"kubernetes.io/projected/8578cb3b-be11-4fd2-8811-e44eaa5f0536-kube-api-access-9n9jk\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.640824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019f8054-04b3-4568-b353-4be58af1ab3b-host-slash\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640825 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-os-release\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-bin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovn-node-metrics-cert\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640913 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-kubelet-config\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-conf-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-etc-kubernetes\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640953 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-netns\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640970 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cnibin\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.640983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-binary-copy\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cnibin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cni-binary-copy\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641093 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-netd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-kubelet\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641149 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/502c5db6-ee40-4043-84d1-b582261cc920-serviceca\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-etc-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-ovn\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641219 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641248 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/019f8054-04b3-4568-b353-4be58af1ab3b-iptables-alerter-script\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjncx\" (UniqueName: \"kubernetes.io/projected/019f8054-04b3-4568-b353-4be58af1ab3b-kube-api-access-fjncx\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.641302 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641279 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b95sj\" (UniqueName: \"kubernetes.io/projected/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-kube-api-access-b95sj\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-k8s-cni-cncf-io\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-kubelet\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-env-overrides\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/502c5db6-ee40-4043-84d1-b582261cc920-host\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnnsk\" (UniqueName: \"kubernetes.io/projected/502c5db6-ee40-4043-84d1-b582261cc920-kube-api-access-dnnsk\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641566 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p68h9\" (UniqueName: \"kubernetes.io/projected/733d8099-0f3f-4195-a5fd-4813d320ce42-kube-api-access-p68h9\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641582 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641604 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-sys-fs\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzxw4\" (UniqueName: \"kubernetes.io/projected/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-kube-api-access-mzxw4\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-slash\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641691 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mpr54\"" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-registration-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641724 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-multus\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.641752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-systemd-units\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-system-cni-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641794 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-socket-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641820 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpgtz\" (UniqueName: \"kubernetes.io/projected/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kube-api-access-mpgtz\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641836 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641842 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-hostroot\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641871 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-bin\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641880 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641895 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-config\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-script-lib\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-device-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.641989 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-97l65\"" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.642001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-netns\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.642024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-dbus\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.642035 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 15:20:14.642206 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.642048 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3c7a6a4-f419-42e8-be7a-f64e2d254290-konnectivity-ca\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.649729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.649704 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 15:20:14.650497 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.650479 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 15:20:14.650598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.650554 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" Jul 12 15:20:14.650654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.650628 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal"] Jul 12 15:20:14.658824 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.658808 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 15:20:14.658907 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.658888 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal"] Jul 12 15:20:14.664311 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.664265 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 15:15:14 +0000 UTC" deadline="2028-04-01 14:54:56.241144348 +0000 UTC" Jul 12 15:20:14.664377 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.664313 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15095h34m41.576835333s" Jul 12 15:20:14.672371 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.672345 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q8mpj" Jul 12 15:20:14.677739 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.677723 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q8mpj" Jul 12 15:20:14.736572 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.736549 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 15:20:14.742237 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-systemd-units\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742237 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-system-cni-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742262 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-socket-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpgtz\" (UniqueName: \"kubernetes.io/projected/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kube-api-access-mpgtz\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742300 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-systemd-units\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-hostroot\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742343 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-system-cni-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-bin\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742391 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-hostroot\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742397 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-config\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-script-lib\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742453 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-socket-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-device-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-netns\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-dbus\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-netns\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742656 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-bin\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742668 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3c7a6a4-f419-42e8-be7a-f64e2d254290-konnectivity-ca\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysconfig\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-os-release\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742791 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-var-lib-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-node-log\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742834 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-device-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.742870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742869 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-var-lib-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742931 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-os-release\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742951 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-node-log\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.742987 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-dbus\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-config\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-system-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743099 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovnkube-script-lib\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-systemd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743066 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-system-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-multus-certs\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743174 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-systemd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743174 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-log-socket\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-multus-certs\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-log-socket\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743210 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3c7a6a4-f419-42e8-be7a-f64e2d254290-agent-certs\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.743529 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.743281 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-conf\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743340 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-systemd\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.743360 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:15.243340939 +0000 UTC m=+1.936619937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-socket-dir-parent\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-daemon-config\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9jk\" (UniqueName: \"kubernetes.io/projected/8578cb3b-be11-4fd2-8811-e44eaa5f0536-kube-api-access-9n9jk\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743452 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-socket-dir-parent\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743457 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019f8054-04b3-4568-b353-4be58af1ab3b-host-slash\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019f8054-04b3-4568-b353-4be58af1ab3b-host-slash\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743492 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-os-release\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743538 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-os-release\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca059fb5-eee1-4767-bb92-d39f488b76a2-tmp-dir\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743586 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3c7a6a4-f419-42e8-be7a-f64e2d254290-konnectivity-ca\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.744339 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-bin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743611 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-bin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovn-node-metrics-cert\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-kubelet-config\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-kubernetes\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-run\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743758 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1bf5087-3ddd-4030-bdca-c5299683ad1d-kubelet-config\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-conf-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743796 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-etc-kubernetes\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743838 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-netns\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743847 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-etc-kubernetes\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cnibin\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743893 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-conf-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cnibin\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-run-netns\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-binary-copy\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-tuned\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.743980 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-daemon-config\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744079 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cnibin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cni-binary-copy\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-netd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-modprobe-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-tmp\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744203 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-multus-cni-dir\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca059fb5-eee1-4767-bb92-d39f488b76a2-hosts-file\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-kubelet\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744292 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744340 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744294 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.745966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744417 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-kubelet\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-var-lib-kubelet\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744472 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-cni-binary-copy\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744495 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/502c5db6-ee40-4043-84d1-b582261cc920-serviceca\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cnibin\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744527 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-sys\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744531 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k854\" (UniqueName: \"kubernetes.io/projected/ca059fb5-eee1-4767-bb92-d39f488b76a2-kube-api-access-7k854\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-cni-netd\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744636 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-etc-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.744625 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-ovn\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.744706 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:15.244691072 +0000 UTC m=+1.937970056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.746565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/019f8054-04b3-4568-b353-4be58af1ab3b-iptables-alerter-script\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744759 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-run-ovn\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744791 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjncx\" (UniqueName: \"kubernetes.io/projected/019f8054-04b3-4568-b353-4be58af1ab3b-kube-api-access-fjncx\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b95sj\" (UniqueName: \"kubernetes.io/projected/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-kube-api-access-b95sj\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744819 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/502c5db6-ee40-4043-84d1-b582261cc920-serviceca\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-host\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744861 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-cni-binary-copy\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-k8s-cni-cncf-io\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744941 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-kubelet\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-env-overrides\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/502c5db6-ee40-4043-84d1-b582261cc920-host\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.744990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnnsk\" (UniqueName: \"kubernetes.io/projected/502c5db6-ee40-4043-84d1-b582261cc920-kube-api-access-dnnsk\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p68h9\" (UniqueName: \"kubernetes.io/projected/733d8099-0f3f-4195-a5fd-4813d320ce42-kube-api-access-p68h9\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-lib-modules\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84v8j\" (UniqueName: \"kubernetes.io/projected/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-kube-api-access-84v8j\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745087 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-sys-fs\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzxw4\" (UniqueName: \"kubernetes.io/projected/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-kube-api-access-mzxw4\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747094 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745122 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-slash\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745120 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-etc-openvswitch\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745181 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-slash\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745307 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/019f8054-04b3-4568-b353-4be58af1ab3b-iptables-alerter-script\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8578cb3b-be11-4fd2-8811-e44eaa5f0536-env-overrides\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-registration-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/502c5db6-ee40-4043-84d1-b582261cc920-host\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-multus\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745413 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8578cb3b-be11-4fd2-8811-e44eaa5f0536-host-kubelet\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745472 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-run-k8s-cni-cncf-io\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-registration-dir\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-host-var-lib-cni-multus\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.745735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-sys-fs\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.747389 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8578cb3b-be11-4fd2-8811-e44eaa5f0536-ovn-node-metrics-cert\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.747761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.747676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3c7a6a4-f419-42e8-be7a-f64e2d254290-agent-certs\") pod \"konnectivity-agent-42264\" (UID: \"d3c7a6a4-f419-42e8-be7a-f64e2d254290\") " pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:14.753364 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.753311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpgtz\" (UniqueName: \"kubernetes.io/projected/59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec-kube-api-access-mpgtz\") pod \"aws-ebs-csi-driver-node-c7wg7\" (UID: \"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:14.754585 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.754566 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:14.754693 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.754594 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:14.754693 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.754609 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:14.754693 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:14.754671 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:15.254652879 +0000 UTC m=+1.947931862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:14.754881 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.754864 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b95sj\" (UniqueName: \"kubernetes.io/projected/b2bc4f44-cbfe-4b48-9157-a80b31cbffea-kube-api-access-b95sj\") pod \"multus-additional-cni-plugins-gktwq\" (UID: \"b2bc4f44-cbfe-4b48-9157-a80b31cbffea\") " pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:14.755090 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.755074 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p68h9\" (UniqueName: \"kubernetes.io/projected/733d8099-0f3f-4195-a5fd-4813d320ce42-kube-api-access-p68h9\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:14.760794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.760770 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzxw4\" (UniqueName: \"kubernetes.io/projected/be136fa6-3d60-4905-9ffb-4a9528a0d7fc-kube-api-access-mzxw4\") pod \"multus-75kzc\" (UID: \"be136fa6-3d60-4905-9ffb-4a9528a0d7fc\") " pod="openshift-multus/multus-75kzc" Jul 12 15:20:14.760998 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.760979 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9jk\" (UniqueName: \"kubernetes.io/projected/8578cb3b-be11-4fd2-8811-e44eaa5f0536-kube-api-access-9n9jk\") pod \"ovnkube-node-58qlj\" (UID: \"8578cb3b-be11-4fd2-8811-e44eaa5f0536\") " pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:14.761360 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.761297 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnnsk\" (UniqueName: \"kubernetes.io/projected/502c5db6-ee40-4043-84d1-b582261cc920-kube-api-access-dnnsk\") pod \"node-ca-b6qnh\" (UID: \"502c5db6-ee40-4043-84d1-b582261cc920\") " pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:14.761360 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:14.761297 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjncx\" (UniqueName: \"kubernetes.io/projected/019f8054-04b3-4568-b353-4be58af1ab3b-kube-api-access-fjncx\") pod \"iptables-alerter-wtbr2\" (UID: \"019f8054-04b3-4568-b353-4be58af1ab3b\") " pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:15.971114 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:15.971068 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a3ddb42b509e449091e6f0bf0e70c5.slice/crio-22a79d9b71c6e851b26083db5b6b608d362a90dc43f768cdf5b199ae2c6626a3 WatchSource:0}: Error finding container 22a79d9b71c6e851b26083db5b6b608d362a90dc43f768cdf5b199ae2c6626a3: Status 404 returned error can't find the container with id 22a79d9b71c6e851b26083db5b6b608d362a90dc43f768cdf5b199ae2c6626a3 Jul 12 15:20:15.971627 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:15.971603 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a325171318a9024f14dff1bce3655e1.slice/crio-05c7034d4d1cba66c5972d66a214cdb1e83f19349bed4377899f5c74d35a3e73 WatchSource:0}: Error finding container 05c7034d4d1cba66c5972d66a214cdb1e83f19349bed4377899f5c74d35a3e73: Status 404 returned error can't find the container with id 05c7034d4d1cba66c5972d66a214cdb1e83f19349bed4377899f5c74d35a3e73 Jul 12 15:20:15.976545 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.976531 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:20:15.989737 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-conf\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.989851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-systemd\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.989851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.989851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca059fb5-eee1-4767-bb92-d39f488b76a2-tmp-dir\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989922 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-systemd\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989920 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysctl-conf\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-kubernetes\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-kubernetes\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.989994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-run\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-tuned\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990058 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-run\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-modprobe-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990104 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-tmp\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca059fb5-eee1-4767-bb92-d39f488b76a2-hosts-file\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:15.990152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990140 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca059fb5-eee1-4767-bb92-d39f488b76a2-tmp-dir\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-var-lib-kubelet\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990189 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca059fb5-eee1-4767-bb92-d39f488b76a2-hosts-file\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-sys\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k854\" (UniqueName: \"kubernetes.io/projected/ca059fb5-eee1-4767-bb92-d39f488b76a2-kube-api-access-7k854\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990261 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-modprobe-d\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-host\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990316 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-lib-modules\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-var-lib-kubelet\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84v8j\" (UniqueName: \"kubernetes.io/projected/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-kube-api-access-84v8j\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990381 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-host\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990456 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-lib-modules\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990463 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-sys\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990473 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysconfig\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.990732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.990541 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-sysconfig\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.992534 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.992518 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-tmp\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.992749 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.992734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-etc-tuned\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.998666 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.998645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84v8j\" (UniqueName: \"kubernetes.io/projected/54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938-kube-api-access-84v8j\") pod \"tuned-ggwwf\" (UID: \"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938\") " pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:15.998666 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:15.998658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k854\" (UniqueName: \"kubernetes.io/projected/ca059fb5-eee1-4767-bb92-d39f488b76a2-kube-api-access-7k854\") pod \"node-resolver-77n27\" (UID: \"ca059fb5-eee1-4767-bb92-d39f488b76a2\") " pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:16.013774 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.013752 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:16.095259 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.095235 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" Jul 12 15:20:16.103465 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.103440 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59bf17e1_3c84_4e69_aa56_5eac8cbfb7ec.slice/crio-5f2359043c913cdeaa1a75da6e6cde0feeec72f45cf0be6498a43f12862256ef WatchSource:0}: Error finding container 5f2359043c913cdeaa1a75da6e6cde0feeec72f45cf0be6498a43f12862256ef: Status 404 returned error can't find the container with id 5f2359043c913cdeaa1a75da6e6cde0feeec72f45cf0be6498a43f12862256ef Jul 12 15:20:16.105144 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.105123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b6qnh" Jul 12 15:20:16.113240 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.113216 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502c5db6_ee40_4043_84d1_b582261cc920.slice/crio-04ab254f2723d2044b72c4f4a37d79ea4ba4995f6d517667e5931b0a03250336 WatchSource:0}: Error finding container 04ab254f2723d2044b72c4f4a37d79ea4ba4995f6d517667e5931b0a03250336: Status 404 returned error can't find the container with id 04ab254f2723d2044b72c4f4a37d79ea4ba4995f6d517667e5931b0a03250336 Jul 12 15:20:16.125747 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.125725 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-75kzc" Jul 12 15:20:16.133133 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.133108 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe136fa6_3d60_4905_9ffb_4a9528a0d7fc.slice/crio-f92e1b35ef76b3ac25deca2e2fc0e00f2b1468e2845655b8a594d58a83e33ffe WatchSource:0}: Error finding container f92e1b35ef76b3ac25deca2e2fc0e00f2b1468e2845655b8a594d58a83e33ffe: Status 404 returned error can't find the container with id f92e1b35ef76b3ac25deca2e2fc0e00f2b1468e2845655b8a594d58a83e33ffe Jul 12 15:20:16.139179 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.139161 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wtbr2" Jul 12 15:20:16.147801 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.147770 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019f8054_04b3_4568_b353_4be58af1ab3b.slice/crio-f94691ce68bcf4167f151c298f67c7eb0d5de87a6877e8c338248d8fbecdacb5 WatchSource:0}: Error finding container f94691ce68bcf4167f151c298f67c7eb0d5de87a6877e8c338248d8fbecdacb5: Status 404 returned error can't find the container with id f94691ce68bcf4167f151c298f67c7eb0d5de87a6877e8c338248d8fbecdacb5 Jul 12 15:20:16.148839 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.148820 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:16.156129 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.156109 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c7a6a4_f419_42e8_be7a_f64e2d254290.slice/crio-122bac678da1a222fde14370ca83532c643ff9a728b12c1e582bd90ab2f0383d WatchSource:0}: Error finding container 122bac678da1a222fde14370ca83532c643ff9a728b12c1e582bd90ab2f0383d: Status 404 returned error can't find the container with id 122bac678da1a222fde14370ca83532c643ff9a728b12c1e582bd90ab2f0383d Jul 12 15:20:16.180254 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.180227 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gktwq" Jul 12 15:20:16.187791 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.187767 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bc4f44_cbfe_4b48_9157_a80b31cbffea.slice/crio-e54ae37bd900f9913e396f80dd3e6384bed45795142246db7caead40ada56f3b WatchSource:0}: Error finding container e54ae37bd900f9913e396f80dd3e6384bed45795142246db7caead40ada56f3b: Status 404 returned error can't find the container with id e54ae37bd900f9913e396f80dd3e6384bed45795142246db7caead40ada56f3b Jul 12 15:20:16.195846 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.195828 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:16.202418 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.202377 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8578cb3b_be11_4fd2_8811_e44eaa5f0536.slice/crio-143e022074ef312bd97ddba9dac665735b45187a32f67cb8585af7ad56437b7d WatchSource:0}: Error finding container 143e022074ef312bd97ddba9dac665735b45187a32f67cb8585af7ad56437b7d: Status 404 returned error can't find the container with id 143e022074ef312bd97ddba9dac665735b45187a32f67cb8585af7ad56437b7d Jul 12 15:20:16.217817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.217795 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" Jul 12 15:20:16.222585 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.222566 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77n27" Jul 12 15:20:16.225536 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.225515 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e2bbcf_b3aa_4f2b_bc0e_39fdf679c938.slice/crio-5bb49c6bc6a75b60be1e3cfba219810a1b9a24ae9909dca8fa64a9a6b9d26d88 WatchSource:0}: Error finding container 5bb49c6bc6a75b60be1e3cfba219810a1b9a24ae9909dca8fa64a9a6b9d26d88: Status 404 returned error can't find the container with id 5bb49c6bc6a75b60be1e3cfba219810a1b9a24ae9909dca8fa64a9a6b9d26d88 Jul 12 15:20:16.230310 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:16.230289 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca059fb5_eee1_4767_bb92_d39f488b76a2.slice/crio-415e3416437aaa155c8511a17b24b79ac79544aad86e048303a8f10ae5cb2a57 WatchSource:0}: Error finding container 415e3416437aaa155c8511a17b24b79ac79544aad86e048303a8f10ae5cb2a57: Status 404 returned error can't find the container with id 415e3416437aaa155c8511a17b24b79ac79544aad86e048303a8f10ae5cb2a57 Jul 12 15:20:16.393256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.393177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:16.393256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.393216 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:16.393500 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.393334 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:16.393500 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.393337 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:16.393500 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.393412 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:17.3933789 +0000 UTC m=+2.943579655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:16.393500 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.393428 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:17.393420734 +0000 UTC m=+2.943621486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:16.493900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.493856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:16.494079 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.493989 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:16.494079 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.494009 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:16.494079 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.494018 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:16.494190 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:16.494081 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:17.494065947 +0000 UTC m=+3.044266701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:16.823024 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.822942 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:15:15 +0000 UTC" deadline="2028-01-13 15:11:14.357826169 +0000 UTC" Jul 12 15:20:16.823024 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.822979 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13199h50m57.534852037s" Jul 12 15:20:16.832330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.832296 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:16.943022 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.942955 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" event={"ID":"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec","Type":"ContainerStarted","Data":"5f2359043c913cdeaa1a75da6e6cde0feeec72f45cf0be6498a43f12862256ef"} Jul 12 15:20:16.946611 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.946535 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" event={"ID":"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938","Type":"ContainerStarted","Data":"5bb49c6bc6a75b60be1e3cfba219810a1b9a24ae9909dca8fa64a9a6b9d26d88"} Jul 12 15:20:16.956523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.956487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerStarted","Data":"e54ae37bd900f9913e396f80dd3e6384bed45795142246db7caead40ada56f3b"} Jul 12 15:20:16.963572 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.963437 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wtbr2" event={"ID":"019f8054-04b3-4568-b353-4be58af1ab3b","Type":"ContainerStarted","Data":"f94691ce68bcf4167f151c298f67c7eb0d5de87a6877e8c338248d8fbecdacb5"} Jul 12 15:20:16.976203 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.976136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"a5a3ddb42b509e449091e6f0bf0e70c5","Type":"ContainerStarted","Data":"22a79d9b71c6e851b26083db5b6b608d362a90dc43f768cdf5b199ae2c6626a3"} Jul 12 15:20:16.982234 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:16.982201 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"5a325171318a9024f14dff1bce3655e1","Type":"ContainerStarted","Data":"05c7034d4d1cba66c5972d66a214cdb1e83f19349bed4377899f5c74d35a3e73"} Jul 12 15:20:17.005159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.005070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77n27" event={"ID":"ca059fb5-eee1-4767-bb92-d39f488b76a2","Type":"ContainerStarted","Data":"415e3416437aaa155c8511a17b24b79ac79544aad86e048303a8f10ae5cb2a57"} Jul 12 15:20:17.013378 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.013330 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"143e022074ef312bd97ddba9dac665735b45187a32f67cb8585af7ad56437b7d"} Jul 12 15:20:17.048878 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.048796 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-42264" event={"ID":"d3c7a6a4-f419-42e8-be7a-f64e2d254290","Type":"ContainerStarted","Data":"122bac678da1a222fde14370ca83532c643ff9a728b12c1e582bd90ab2f0383d"} Jul 12 15:20:17.054693 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.054659 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-75kzc" event={"ID":"be136fa6-3d60-4905-9ffb-4a9528a0d7fc","Type":"ContainerStarted","Data":"f92e1b35ef76b3ac25deca2e2fc0e00f2b1468e2845655b8a594d58a83e33ffe"} Jul 12 15:20:17.068693 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.068661 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b6qnh" event={"ID":"502c5db6-ee40-4043-84d1-b582261cc920","Type":"ContainerStarted","Data":"04ab254f2723d2044b72c4f4a37d79ea4ba4995f6d517667e5931b0a03250336"} Jul 12 15:20:17.402984 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.402944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:17.403180 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.403019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:17.403180 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.403175 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:17.403296 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.403235 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:19.403217895 +0000 UTC m=+4.953418672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:17.403681 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.403658 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:17.403783 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.403718 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:19.403700817 +0000 UTC m=+4.953901592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:17.504453 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.504417 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:17.504621 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.504604 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:17.504693 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.504629 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:17.504693 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.504642 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:17.504754 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.504700 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:19.504681351 +0000 UTC m=+5.054882125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:17.823938 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.823571 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:15:15 +0000 UTC" deadline="2028-04-01 09:40:26.333954914 +0000 UTC" Jul 12 15:20:17.823938 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.823857 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15090h20m8.510105933s" Jul 12 15:20:17.924056 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.924019 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:17.924239 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.924152 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:17.924654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.924633 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:17.924753 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.924731 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:17.924849 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.924832 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:17.924945 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:17.924928 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:17.951050 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:17.950772 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:19.421665 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.421621 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:19.422129 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.421699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:19.422129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.421883 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:19.422129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.421948 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:23.421928554 +0000 UTC m=+8.972129317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:19.422582 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.422352 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:19.422582 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.422426 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:23.422390607 +0000 UTC m=+8.972591369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:19.523300 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.523261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:19.523482 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.523451 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:19.523482 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.523476 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:19.523595 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.523489 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:19.523595 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.523552 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:23.523531233 +0000 UTC m=+9.073732010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:19.924327 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.924291 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:19.924535 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.924467 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:19.924610 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.924554 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:19.925008 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:19.924986 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:19.925111 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.925084 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:19.925185 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:19.925166 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:21.924670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:21.924831 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:21.924957 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:21.925067 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:21.925150 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:21.925619 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:21.925231 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:23.458160 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.458124 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:23.458598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.458184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:23.458598 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.458322 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:23.458598 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.458384 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:31.458364032 +0000 UTC m=+17.008564789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:23.458814 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.458799 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:23.458853 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.458842 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:31.458830989 +0000 UTC m=+17.009031742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:23.559200 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.559158 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:23.559448 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.559363 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:23.559448 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.559386 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:23.559448 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.559416 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:23.559671 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.559468 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:31.559454653 +0000 UTC m=+17.109655405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:23.923893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.923860 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:23.924050 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.923903 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:23.924050 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:23.923905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:23.924050 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.923979 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:23.924216 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.924098 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:23.924216 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:23.924181 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:25.924548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:25.924674 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:25.925069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:25.925170 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:25.925233 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:25.925358 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:25.925313 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:27.924218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:27.924181 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:27.924218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:27.924229 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:27.924695 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:27.924231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:27.924695 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:27.924302 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:27.924695 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:27.924482 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:27.924695 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:27.924602 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:29.923719 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:29.923679 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:29.924122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:29.923729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:29.924122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:29.923681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:29.924122 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:29.923799 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:29.924122 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:29.923858 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:29.924122 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:29.923937 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:30.927056 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:30.927031 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:30.927576 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:30.927144 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:31.525087 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:31.525043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:31.525362 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:31.525140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:31.525362 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.525172 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:31.525362 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.525226 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:31.525362 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.525248 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:47.525228068 +0000 UTC m=+33.075428824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:31.525362 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.525271 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:20:47.525257952 +0000 UTC m=+33.075458709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:31.625496 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:31.625465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:31.625678 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.625658 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:31.625717 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.625687 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:31.625717 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.625701 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:31.625790 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.625758 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:20:47.625744486 +0000 UTC m=+33.175945239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:31.923642 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:31.923601 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:31.923642 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:31.923627 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:31.923863 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.923729 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:31.923945 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:31.923864 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:32.924228 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:32.924199 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:32.924635 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:32.924309 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:33.924662 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:33.924623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:33.925105 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:33.924670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:33.925810 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:33.925776 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:33.925895 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:33.925731 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:34.924410 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:34.924367 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:34.924554 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:34.924469 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:35.924686 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:35.924643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:35.925104 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:35.924704 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:35.925104 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:35.924774 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:35.925104 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:35.924906 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:36.924390 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:36.924367 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:36.924526 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:36.924496 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:37.924263 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:37.924177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:37.925279 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:37.924331 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:37.925279 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:37.925052 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:37.925279 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:37.925164 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:38.117264 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117104 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:20:38.117727 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117708 2563 generic.go:356] "Generic (PLEG): container finished" podID="8578cb3b-be11-4fd2-8811-e44eaa5f0536" containerID="ce9f0d0538f572dfd1a04cc4cebb1554af9198dab30228e6b1a819384884bb02" exitCode=1 Jul 12 15:20:38.117802 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"95e404a6a0db1626ed8d5230102dc24f04bc9ff7a5ed3ed7ea195275e144661b"} Jul 12 15:20:38.117887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117872 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"1cdb1625434f6ff8cfc11e2ef189c15380aec6d9dd67c7d63255a0b8fc39072f"} Jul 12 15:20:38.117941 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117929 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"fdad06e11baef051f226c4c32d89414267c45af92eaabb33ff4f87f8c62a1b18"} Jul 12 15:20:38.117984 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.117975 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"e976f25dad3a2086741ab9e9d36594d293d343998b98e85c3f408d9fc8f1ce7a"} Jul 12 15:20:38.118036 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.118007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerDied","Data":"ce9f0d0538f572dfd1a04cc4cebb1554af9198dab30228e6b1a819384884bb02"} Jul 12 15:20:38.118085 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.118040 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"4497bb1f885676c193725c4c769144ae30b17bb8a49459817d406899932f21a9"} Jul 12 15:20:38.119142 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.119124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-75kzc" event={"ID":"be136fa6-3d60-4905-9ffb-4a9528a0d7fc","Type":"ContainerStarted","Data":"e3d859c918efaab70ad407453e420a148fc0b02ee2180f91f1327ffd74eff7c1"} Jul 12 15:20:38.120544 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.120514 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" event={"ID":"54e2bbcf-b3aa-4f2b-bc0e-39fdf679c938","Type":"ContainerStarted","Data":"c04beafb23fe67aeb62d38bf91e69ae4830d8367774921b9e4e41b5314b83c59"} Jul 12 15:20:38.121719 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.121700 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" event={"ID":"a5a3ddb42b509e449091e6f0bf0e70c5","Type":"ContainerStarted","Data":"7ea165b6cd75899d200dc8ccf83ee188019544b8873998b5280a17f4275b49dc"} Jul 12 15:20:38.155172 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.155117 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ggwwf" podStartSLOduration=2.338678957 podStartE2EDuration="23.155097406s" totalImagesPullingTime="20.816418449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:38.154786344 +0000 UTC m=+23.704987137" watchObservedRunningTime="2026-07-12 15:20:38.155097406 +0000 UTC m=+23.705298181" Jul 12 15:20:38.155494 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.155293 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-75kzc" podStartSLOduration=2.214612929 podStartE2EDuration="23.155287139s" totalImagesPullingTime="20.94067421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:38.135910216 +0000 UTC m=+23.686110991" watchObservedRunningTime="2026-07-12 15:20:38.155287139 +0000 UTC m=+23.705487915" Jul 12 15:20:38.170734 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.170667 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-230.ec2.internal" podStartSLOduration=23.170643964 podStartE2EDuration="23.170643964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:38.170553477 +0000 UTC m=+23.720754272" watchObservedRunningTime="2026-07-12 15:20:38.170643964 +0000 UTC m=+23.720844741" Jul 12 15:20:38.924049 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:38.924009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:38.924301 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:38.924151 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:39.125598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.125559 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" event={"ID":"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec","Type":"ContainerStarted","Data":"13e1cf2d286df64caa7841b394768417865e9e364b8441da6009c3f5cfbf7145"} Jul 12 15:20:39.127240 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.127214 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="7c775db52fd1ca9a97d6aa7f29bbc7afa6b6ed6c45861e8c201e2b12c3fa15e9" exitCode=0 Jul 12 15:20:39.127354 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.127308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"7c775db52fd1ca9a97d6aa7f29bbc7afa6b6ed6c45861e8c201e2b12c3fa15e9"} Jul 12 15:20:39.129003 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.128946 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wtbr2" event={"ID":"019f8054-04b3-4568-b353-4be58af1ab3b","Type":"ContainerStarted","Data":"9751338a3e808e67244640932ea2bce6a472512baccc7e94022d755153f9e3a1"} Jul 12 15:20:39.130625 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.130603 2563 generic.go:356] "Generic (PLEG): container finished" podID="5a325171318a9024f14dff1bce3655e1" containerID="2fe92943d0892c87b4cc205e0da03482c64a3765b824f8fae328b6e13cab2f14" exitCode=0 Jul 12 15:20:39.130736 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.130674 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"5a325171318a9024f14dff1bce3655e1","Type":"ContainerDied","Data":"2fe92943d0892c87b4cc205e0da03482c64a3765b824f8fae328b6e13cab2f14"} Jul 12 15:20:39.133502 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.133478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77n27" event={"ID":"ca059fb5-eee1-4767-bb92-d39f488b76a2","Type":"ContainerStarted","Data":"00f0999541ebd4edb76c530d8214a3607220a2af759e1e2030886efb4212f807"} Jul 12 15:20:39.135548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.135493 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-42264" event={"ID":"d3c7a6a4-f419-42e8-be7a-f64e2d254290","Type":"ContainerStarted","Data":"7c530e82c3c3662447b3cbd97e8d5066419ede66b07405926861100623f7de3e"} Jul 12 15:20:39.136957 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.136935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b6qnh" event={"ID":"502c5db6-ee40-4043-84d1-b582261cc920","Type":"ContainerStarted","Data":"33582716f65403cc0aba9e05027191bb5024f5beb6d43ac497d61c35a2e63f10"} Jul 12 15:20:39.165512 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.165445 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b6qnh" podStartSLOduration=3.238305386 podStartE2EDuration="24.165426128s" totalImagesPullingTime="20.927120742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:39.164635674 +0000 UTC m=+24.714836447" watchObservedRunningTime="2026-07-12 15:20:39.165426128 +0000 UTC m=+24.715627273" Jul 12 15:20:39.178579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.178486 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wtbr2" podStartSLOduration=3.285990856 podStartE2EDuration="24.178469334s" totalImagesPullingTime="20.892478478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:39.178037858 +0000 UTC m=+24.728238632" watchObservedRunningTime="2026-07-12 15:20:39.178469334 +0000 UTC m=+24.728670108" Jul 12 15:20:39.195296 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.195247 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-77n27" podStartSLOduration=3.384918936 podStartE2EDuration="24.19523274s" totalImagesPullingTime="20.810313804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:39.195148293 +0000 UTC m=+24.745349067" watchObservedRunningTime="2026-07-12 15:20:39.19523274 +0000 UTC m=+24.745433515" Jul 12 15:20:39.224816 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.224769 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-42264" podStartSLOduration=3.340453829 podStartE2EDuration="24.224751942s" totalImagesPullingTime="20.884298113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:39.224206534 +0000 UTC m=+24.774407308" watchObservedRunningTime="2026-07-12 15:20:39.224751942 +0000 UTC m=+24.774952756" Jul 12 15:20:39.620343 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.620310 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 15:20:39.856860 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.856718 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T15:20:39.620328166Z","UUID":"4ad051c6-b4e9-4c95-8d49-3745cd102f2e","Handler":null,"Name":"","Endpoint":""} Jul 12 15:20:39.858146 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.858122 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 15:20:39.858251 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.858161 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 15:20:39.924705 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.924667 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:39.924705 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:39.924694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:39.925155 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:39.924787 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:39.925155 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:39.924927 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:40.141654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.141570 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" event={"ID":"5a325171318a9024f14dff1bce3655e1","Type":"ContainerStarted","Data":"beb8b32712b074c7e3d3a0a40b902ad3643b6246d99c8e35f9895ed98a9276b6"} Jul 12 15:20:40.145622 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.145574 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:20:40.146043 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.146017 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"126b7b87b057ec21bc18912556d4d570f93497b7ee768ac67d630a88ddd6d358"} Jul 12 15:20:40.148587 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.148563 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" event={"ID":"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec","Type":"ContainerStarted","Data":"7192797b80518ed228ce8ac51dd9930a6baf0e7f3c31cd1d74ed18b6a85a705a"} Jul 12 15:20:40.156846 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.156786 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-230.ec2.internal" podStartSLOduration=25.156769792 podStartE2EDuration="25.156769792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:40.156321653 +0000 UTC m=+25.706522429" watchObservedRunningTime="2026-07-12 15:20:40.156769792 +0000 UTC m=+25.706970568" Jul 12 15:20:40.924719 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:40.924683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:40.925279 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:40.924810 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:41.153354 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:41.153102 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" event={"ID":"59bf17e1-3c84-4e69-aa56-5eac8cbfb7ec","Type":"ContainerStarted","Data":"00541b9c7ce58f09f7b6b1ac73d55a3a8240a0a32c26d965839a518ce13be49a"} Jul 12 15:20:41.170811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:41.170759 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7wg7" podStartSLOduration=3.573890467 podStartE2EDuration="27.170744712s" totalImagesPullingTime="23.596854245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:41.170408371 +0000 UTC m=+26.720609159" watchObservedRunningTime="2026-07-12 15:20:41.170744712 +0000 UTC m=+26.720945488" Jul 12 15:20:41.924584 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:41.924548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:41.924779 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:41.924548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:41.924779 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:41.924696 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:41.924779 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:41.924752 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:42.924221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:42.924023 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:42.924426 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:42.924321 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:43.162266 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.162238 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:20:43.163063 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.162649 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"c9cdb2d013940a8608734679a8b1a0c19b6adf249669788aa5512de71909ff21"} Jul 12 15:20:43.163063 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.162946 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:43.163228 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.163206 2563 scope.go:117] "RemoveContainer" containerID="ce9f0d0538f572dfd1a04cc4cebb1554af9198dab30228e6b1a819384884bb02" Jul 12 15:20:43.181654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.181626 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:43.924036 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.924002 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:43.924368 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.924019 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:43.924368 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:43.924160 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:43.924368 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:43.924257 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:43.977633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.977601 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:43.978168 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:43.978147 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:44.131807 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.131777 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h7cqj"] Jul 12 15:20:44.140035 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.140009 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dtq8d"] Jul 12 15:20:44.141011 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.140959 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9fw9"] Jul 12 15:20:44.141499 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.141479 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:44.141613 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:44.141578 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:44.167476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.167441 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="52ba2eb8feb7d5a263be22a99248d23272067aac4b4b3513a780c2d73a176c66" exitCode=0 Jul 12 15:20:44.167946 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.167524 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"52ba2eb8feb7d5a263be22a99248d23272067aac4b4b3513a780c2d73a176c66"} Jul 12 15:20:44.172858 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.172841 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:20:44.173222 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173203 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:44.173324 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173241 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" event={"ID":"8578cb3b-be11-4fd2-8811-e44eaa5f0536","Type":"ContainerStarted","Data":"2658fa3c32e658a31411cf8cd7a845b90dfed0728ce5963719d600a13e8ebabe"} Jul 12 15:20:44.173324 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173305 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:44.173449 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173433 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:44.173593 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:44.173554 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:44.173703 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:44.173684 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:44.173750 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173716 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:44.173750 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173739 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:44.173908 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.173896 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-42264" Jul 12 15:20:44.189698 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.189675 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:20:44.209651 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:44.209602 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" podStartSLOduration=8.267780731 podStartE2EDuration="29.209585817s" totalImagesPullingTime="20.941805086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:44.208945597 +0000 UTC m=+29.759146373" watchObservedRunningTime="2026-07-12 15:20:44.209585817 +0000 UTC m=+29.759786593" Jul 12 15:20:45.923937 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:45.923906 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:45.924319 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:45.923993 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:45.924319 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:45.923999 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:45.924319 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:45.924018 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:45.924319 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:45.924087 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:45.924319 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:45.924152 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:46.180507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:46.180442 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="51173bfeef240f602276df9b45f8ea0ff855a0ba85aa1eff5dd72a7db78ca95d" exitCode=0 Jul 12 15:20:46.180604 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:46.180529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"51173bfeef240f602276df9b45f8ea0ff855a0ba85aa1eff5dd72a7db78ca95d"} Jul 12 15:20:47.542574 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.542360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:47.543129 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.542619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:47.543129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.542558 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:47.543129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.542855 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret podName:a1bf5087-3ddd-4030-bdca-c5299683ad1d nodeName:}" failed. No retries permitted until 2026-07-12 15:21:19.542835107 +0000 UTC m=+65.093035861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret") pod "global-pull-secret-syncer-h7cqj" (UID: "a1bf5087-3ddd-4030-bdca-c5299683ad1d") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:47.543129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.542770 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:47.543129 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.542939 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:19.542918897 +0000 UTC m=+65.093119650 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:47.643510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.643474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:47.643677 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.643665 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:47.643725 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.643687 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:47.643725 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.643701 2563 projected.go:196] Error preparing data for projected volume kube-api-access-rll6h for pod openshift-network-diagnostics/network-check-target-l9fw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:47.643796 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.643757 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h podName:5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf nodeName:}" failed. No retries permitted until 2026-07-12 15:21:19.643742339 +0000 UTC m=+65.193943093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rll6h" (UniqueName: "kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h") pod "network-check-target-l9fw9" (UID: "5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:47.924694 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.924651 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:47.924694 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.924689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:47.924694 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:47.924699 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:47.924925 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.924782 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:47.924925 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.924894 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:47.925005 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:47.924989 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:48.186821 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:48.186732 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="00f286d034d59998f6979a46944c7e5c2376e5736112e07992d3c30937551170" exitCode=0 Jul 12 15:20:48.186821 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:48.186788 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"00f286d034d59998f6979a46944c7e5c2376e5736112e07992d3c30937551170"} Jul 12 15:20:49.924126 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:49.924091 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:49.924745 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:49.924092 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:49.924745 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:49.924214 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:49.924745 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:49.924217 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h7cqj" podUID="a1bf5087-3ddd-4030-bdca-c5299683ad1d" Jul 12 15:20:49.924745 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:49.924296 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9fw9" podUID="5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf" Jul 12 15:20:49.924745 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:49.924385 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:20:50.285057 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.284985 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-230.ec2.internal" event="NodeReady" Jul 12 15:20:50.285231 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.285147 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 15:20:50.321429 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.319280 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-2gmjr"] Jul 12 15:20:50.325290 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.325268 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:20:50.325466 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.325415 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.328028 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.328005 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 12 15:20:50.328028 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.328010 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qx8d5\"" Jul 12 15:20:50.328214 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.328010 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 12 15:20:50.330167 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.330144 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252"] Jul 12 15:20:50.330279 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.330245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.335608 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.335582 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh"] Jul 12 15:20:50.336797 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.336755 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 15:20:50.341512 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.341471 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq"] Jul 12 15:20:50.342531 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.342508 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.345469 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.345919 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.346083 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-2gmjr"] Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.346135 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252"] Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.346286 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hhn57\"" Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.346307 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.347746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.347260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.349522 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.348672 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 15:20:50.349522 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.348731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 15:20:50.349522 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.348936 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 12 15:20:50.349522 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.348977 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wn7g5\"" Jul 12 15:20:50.349522 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.349063 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 15:20:50.350459 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.350439 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq"] Jul 12 15:20:50.351822 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.351806 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 15:20:50.352496 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352478 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 12 15:20:50.352771 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352740 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 12 15:20:50.352840 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352653 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh"] Jul 12 15:20:50.352840 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352828 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:20:50.352961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352857 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ckjzp"] Jul 12 15:20:50.352961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352703 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 12 15:20:50.352961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352755 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 15:20:50.353228 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.352614 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 12 15:20:50.359240 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.359218 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xgtjd"] Jul 12 15:20:50.359731 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.359712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.361491 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.361471 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgtjd"] Jul 12 15:20:50.361602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.361505 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ckjzp"] Jul 12 15:20:50.361735 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.361603 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.362495 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.362481 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 15:20:50.362685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.362665 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ns4rd\"" Jul 12 15:20:50.362685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.362678 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 15:20:50.364358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.364338 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-79jmw\"" Jul 12 15:20:50.364600 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.364584 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 15:20:50.364789 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.364773 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 15:20:50.365016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.364996 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 15:20:50.461682 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.461682 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461686 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tl6r\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461824 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e9db2933-abb6-488a-b9c4-15daa3a4c734-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461842 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461862 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbn9m\" (UniqueName: \"kubernetes.io/projected/9f686a4e-6106-4ce9-8438-fefae0571b84-kube-api-access-vbn9m\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.461915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0733aa5-675c-41c8-b924-87c2b65fb004-config-volume\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9db2933-abb6-488a-b9c4-15daa3a4c734-tmp\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.461988 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/efb84714-145d-4faa-b47d-87a8c2691518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462169 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9qd\" (UniqueName: \"kubernetes.io/projected/f0733aa5-675c-41c8-b924-87c2b65fb004-kube-api-access-6b9qd\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462173 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462252 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ea66087-0992-41de-87d9-59515e7eb1d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0733aa5-675c-41c8-b924-87c2b65fb004-tmp-dir\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjfb8\" (UniqueName: \"kubernetes.io/projected/e9db2933-abb6-488a-b9c4-15daa3a4c734-kube-api-access-sjfb8\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.462448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f686a4e-6106-4ce9-8438-fefae0571b84-cert\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462504 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjlz\" (UniqueName: \"kubernetes.io/projected/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-kube-api-access-rdjlz\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.462780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.462563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5k9v\" (UniqueName: \"kubernetes.io/projected/efb84714-145d-4faa-b47d-87a8c2691518-kube-api-access-x5k9v\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.563461 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563368 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.563461 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563467 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9qd\" (UniqueName: \"kubernetes.io/projected/f0733aa5-675c-41c8-b924-87c2b65fb004-kube-api-access-6b9qd\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ea66087-0992-41de-87d9-59515e7eb1d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.563534 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0733aa5-675c-41c8-b924-87c2b65fb004-tmp-dir\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.563626 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:51.063605594 +0000 UTC m=+36.613806348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:20:50.563678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjfb8\" (UniqueName: \"kubernetes.io/projected/e9db2933-abb6-488a-b9c4-15daa3a4c734-kube-api-access-sjfb8\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563683 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563742 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f686a4e-6106-4ce9-8438-fefae0571b84-cert\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563838 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjlz\" (UniqueName: \"kubernetes.io/projected/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-kube-api-access-rdjlz\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563899 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5k9v\" (UniqueName: \"kubernetes.io/projected/efb84714-145d-4faa-b47d-87a8c2691518-kube-api-access-x5k9v\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563933 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0733aa5-675c-41c8-b924-87c2b65fb004-tmp-dir\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.563991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.564006 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tl6r\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.564043 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:51.064030674 +0000 UTC m=+36.614231429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e9db2933-abb6-488a-b9c4-15daa3a4c734-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbn9m\" (UniqueName: \"kubernetes.io/projected/9f686a4e-6106-4ce9-8438-fefae0571b84-kube-api-access-vbn9m\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564180 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564187 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0733aa5-675c-41c8-b924-87c2b65fb004-config-volume\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9db2933-abb6-488a-b9c4-15daa3a4c734-tmp\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/efb84714-145d-4faa-b47d-87a8c2691518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.564829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.565096 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.564842 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.565258 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9db2933-abb6-488a-b9c4-15daa3a4c734-tmp\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.565471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4ea66087-0992-41de-87d9-59515e7eb1d6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.565550 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.566304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.566499 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.566513 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:20:50.566924 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:50.566562 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:51.066546903 +0000 UTC m=+36.616747656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:20:50.567743 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.567699 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0733aa5-675c-41c8-b924-87c2b65fb004-config-volume\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.569494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.569905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.570459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/efb84714-145d-4faa-b47d-87a8c2691518-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.570648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-ca\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.570665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-hub\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.570788 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.570875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.571972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f686a4e-6106-4ce9-8438-fefae0571b84-cert\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.571989 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e9db2933-abb6-488a-b9c4-15daa3a4c734-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.573580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjlz\" (UniqueName: \"kubernetes.io/projected/9f5c2f9f-c928-400a-b6f9-15e663c2e9c2-kube-api-access-rdjlz\") pod \"cluster-proxy-proxy-agent-56ff8f58bb-lnwwh\" (UID: \"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.573811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tl6r\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.574988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9qd\" (UniqueName: \"kubernetes.io/projected/f0733aa5-675c-41c8-b924-87c2b65fb004-kube-api-access-6b9qd\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.575801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5k9v\" (UniqueName: \"kubernetes.io/projected/efb84714-145d-4faa-b47d-87a8c2691518-kube-api-access-x5k9v\") pod \"managed-serviceaccount-addon-agent-7fbb6589d4-mp252\" (UID: \"efb84714-145d-4faa-b47d-87a8c2691518\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.575987 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.577081 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjfb8\" (UniqueName: \"kubernetes.io/projected/e9db2933-abb6-488a-b9c4-15daa3a4c734-kube-api-access-sjfb8\") pod \"klusterlet-addon-workmgr-6c59565cf5-hd4vq\" (UID: \"e9db2933-abb6-488a-b9c4-15daa3a4c734\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.578434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.577470 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbn9m\" (UniqueName: \"kubernetes.io/projected/9f686a4e-6106-4ce9-8438-fefae0571b84-kube-api-access-vbn9m\") pod \"ingress-canary-xgtjd\" (UID: \"9f686a4e-6106-4ce9-8438-fefae0571b84\") " pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.678612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.678569 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:20:50.685939 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.685917 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:20:50.692963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.692937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:20:50.718563 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.718486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xgtjd" Jul 12 15:20:50.897625 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.897344 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq"] Jul 12 15:20:50.905717 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.905692 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh"] Jul 12 15:20:50.909625 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:50.909586 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f5c2f9f_c928_400a_b6f9_15e663c2e9c2.slice/crio-a3a2e9b46e966d05fdfc9768226a331b212982b81037b9621ae3ecceed148827 WatchSource:0}: Error finding container a3a2e9b46e966d05fdfc9768226a331b212982b81037b9621ae3ecceed148827: Status 404 returned error can't find the container with id a3a2e9b46e966d05fdfc9768226a331b212982b81037b9621ae3ecceed148827 Jul 12 15:20:50.917993 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.917971 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252"] Jul 12 15:20:50.921049 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:50.921015 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefb84714_145d_4faa_b47d_87a8c2691518.slice/crio-f81d19d69c231b64b5d788be2f6b259cea63602fec3bc435990ad19aff4986fc WatchSource:0}: Error finding container f81d19d69c231b64b5d788be2f6b259cea63602fec3bc435990ad19aff4986fc: Status 404 returned error can't find the container with id f81d19d69c231b64b5d788be2f6b259cea63602fec3bc435990ad19aff4986fc Jul 12 15:20:50.921219 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:50.921201 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xgtjd"] Jul 12 15:20:50.925026 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:20:50.925001 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f686a4e_6106_4ce9_8438_fefae0571b84.slice/crio-2d080d1c4d035c0e1c10484df8783b553fb26cbb60a43a4b0bf842ef7bddd1db WatchSource:0}: Error finding container 2d080d1c4d035c0e1c10484df8783b553fb26cbb60a43a4b0bf842ef7bddd1db: Status 404 returned error can't find the container with id 2d080d1c4d035c0e1c10484df8783b553fb26cbb60a43a4b0bf842ef7bddd1db Jul 12 15:20:51.074191 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.074103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:51.074191 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.074165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.074237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074263 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074287 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074310 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074347 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:52.074328466 +0000 UTC m=+37.624529221 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074363 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:52.074356258 +0000 UTC m=+37.624557011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:20:51.074388 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074362 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:20:51.074889 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:51.074443 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:52.074426489 +0000 UTC m=+37.624627249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:20:51.155924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.155884 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:20:51.196752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.196712 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgtjd" event={"ID":"9f686a4e-6106-4ce9-8438-fefae0571b84","Type":"ContainerStarted","Data":"2d080d1c4d035c0e1c10484df8783b553fb26cbb60a43a4b0bf842ef7bddd1db"} Jul 12 15:20:51.197981 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.197955 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerStarted","Data":"a3a2e9b46e966d05fdfc9768226a331b212982b81037b9621ae3ecceed148827"} Jul 12 15:20:51.199335 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.199302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerStarted","Data":"4f7e571dde18b158568af25a276b2f323740226bc2aefb20293a1a1598d4d27a"} Jul 12 15:20:51.200466 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.200444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerStarted","Data":"f81d19d69c231b64b5d788be2f6b259cea63602fec3bc435990ad19aff4986fc"} Jul 12 15:20:51.238687 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.238646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:20:51.924261 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.924224 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:20:51.924261 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.924249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:20:51.924545 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.924240 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:20:51.927135 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.927112 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 15:20:51.928355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.928332 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 15:20:51.928502 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.928366 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ztf2\"" Jul 12 15:20:51.928502 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.928459 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bw5ll\"" Jul 12 15:20:51.928821 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.928799 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 15:20:51.928966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:51.928862 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 15:20:52.082739 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:52.082703 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:52.082966 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.082884 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:20:52.082966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:52.082934 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:52.082966 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.082946 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:54.082926846 +0000 UTC m=+39.633127601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:20:52.083133 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:52.083000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:52.083133 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.083067 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:20:52.083133 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.083083 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:20:52.083133 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.083095 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:52.083133 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.083131 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:54.083114065 +0000 UTC m=+39.633314839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:20:52.083372 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:52.083149 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:54.083139722 +0000 UTC m=+39.633340480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:20:53.212481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:53.211092 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="0e938b41ba967f837ca3339b7367d28a360aa5fec030d62dcc06033467fc265f" exitCode=0 Jul 12 15:20:53.212481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:53.211260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"0e938b41ba967f837ca3339b7367d28a360aa5fec030d62dcc06033467fc265f"} Jul 12 15:20:54.100849 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:54.100540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:54.101010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:54.100892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:54.101010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:54.100973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101146 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101219 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:58.101196884 +0000 UTC m=+43.651397653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101670 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101725 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:58.10170858 +0000 UTC m=+43.651909335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101795 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101807 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:20:54.101989 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:54.101843 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:58.101830605 +0000 UTC m=+43.652031376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:20:54.227120 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:54.226112 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2bc4f44-cbfe-4b48-9157-a80b31cbffea" containerID="b614e44b7b41bb1b64fe1205ccc6b82f6633b4e4a0c2ff91c123bc1aa415a8cc" exitCode=0 Jul 12 15:20:54.227120 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:54.226177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerDied","Data":"b614e44b7b41bb1b64fe1205ccc6b82f6633b4e4a0c2ff91c123bc1aa415a8cc"} Jul 12 15:20:55.629763 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:55.629691 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:20:58.135180 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:58.135141 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:58.135190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:58.135252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135311 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135331 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135367 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135367 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135415 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:06.135373403 +0000 UTC m=+51.685574156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135439 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:06.135423441 +0000 UTC m=+51.685624196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:20:58.135738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:20:58.135459 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:06.135447661 +0000 UTC m=+51.685648414 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:20:58.999010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:58.998971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:20:59.091025 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:20:59.090979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:00.237040 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:00.237004 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:00.287505 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:00.287479 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:01.248158 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.248122 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xgtjd" event={"ID":"9f686a4e-6106-4ce9-8438-fefae0571b84","Type":"ContainerStarted","Data":"6db7d17522c2c9be63d86c03574a7216066a28ed2c33106df6d0309d0a3ac453"} Jul 12 15:21:01.250092 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.250073 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerStarted","Data":"ae3274f56044f36af9eadaeb877257281776075f26a70fb348b9091cbaafb45c"} Jul 12 15:21:01.250188 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.250103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerStarted","Data":"b1c407f671a6776119f6646409be733def1e5b5701d322a792c20161a011b6f8"} Jul 12 15:21:01.250188 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.250120 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerStarted","Data":"41968ee2d4e34769ecc80dfacd313fe9c3220392b41c593259f02cb14c2c261d"} Jul 12 15:21:01.251565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.251534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerStarted","Data":"290fa4996ceef10e6e5d4890796bd71fea85a1500277f68d45254c0892ea2cdd"} Jul 12 15:21:01.251656 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.251585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:21:01.253505 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.253480 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:21:01.254814 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.254795 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gktwq" event={"ID":"b2bc4f44-cbfe-4b48-9157-a80b31cbffea","Type":"ContainerStarted","Data":"125fce67504e19f137fa98209b54274dd224c8f1120a202bddfc8234b88207f9"} Jul 12 15:21:01.256159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.256134 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerStarted","Data":"5413187def2ec5ede7b254003dbe31f06818092068249dcdc02daf380083b92c"} Jul 12 15:21:01.267610 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.267574 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xgtjd" podStartSLOduration=1.875593487 podStartE2EDuration="11.267561518s" totalImagesPullingTime="9.391968031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:01.267093701 +0000 UTC m=+46.817294492" watchObservedRunningTime="2026-07-12 15:21:01.267561518 +0000 UTC m=+46.817762292" Jul 12 15:21:01.280827 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.280785 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" podStartSLOduration=6.884696046 podStartE2EDuration="16.280771447s" totalImagesPullingTime="9.396075401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:01.280108746 +0000 UTC m=+46.830309522" watchObservedRunningTime="2026-07-12 15:21:01.280771447 +0000 UTC m=+46.830972226" Jul 12 15:21:01.296750 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.296654 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" podStartSLOduration=6.8760649780000005 podStartE2EDuration="16.296637615s" totalImagesPullingTime="9.420572637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:01.296449043 +0000 UTC m=+46.846649843" watchObservedRunningTime="2026-07-12 15:21:01.296637615 +0000 UTC m=+46.846838390" Jul 12 15:21:01.316384 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.316338 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gktwq" podStartSLOduration=13.582413698 podStartE2EDuration="46.31631739s" totalImagesPullingTime="32.733903692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:01.314918726 +0000 UTC m=+46.865119500" watchObservedRunningTime="2026-07-12 15:21:01.31631739 +0000 UTC m=+46.866518166" Jul 12 15:21:01.332281 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.332237 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" podStartSLOduration=6.90835315 podStartE2EDuration="16.332224066s" totalImagesPullingTime="9.423870916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:01.33184202 +0000 UTC m=+46.882042796" watchObservedRunningTime="2026-07-12 15:21:01.332224066 +0000 UTC m=+46.882424819" Jul 12 15:21:01.435900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.435850 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:01.535755 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:01.535671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:02.270280 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:02.270252 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:02.326076 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:02.326044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:02.593014 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:02.592924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:02.652870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:02.652813 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:04.447269 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:04.447218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:04.497118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:04.497085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:06.201117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:06.201074 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:06.201168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:06.201209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201237 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201305 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201319 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201328 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:22.201306676 +0000 UTC m=+67.751507434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201338 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201351 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:22.201340426 +0000 UTC m=+67.751541201 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:21:06.201603 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:06.201388 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:22.201372288 +0000 UTC m=+67.751573044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:21:16.198742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:16.198717 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-58qlj" Jul 12 15:21:16.354503 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:16.354469 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:16.453300 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:16.453222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:19.607211 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.607157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:21:19.607631 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.607254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:21:19.609752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.609729 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 15:21:19.609884 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.609869 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 15:21:19.617815 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:19.617797 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:21:19.617878 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:19.617853 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:22:23.617836538 +0000 UTC m=+129.168037290 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : secret "metrics-daemon-secret" not found Jul 12 15:21:19.621806 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.621776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1bf5087-3ddd-4030-bdca-c5299683ad1d-original-pull-secret\") pod \"global-pull-secret-syncer-h7cqj\" (UID: \"a1bf5087-3ddd-4030-bdca-c5299683ad1d\") " pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:21:19.707907 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.707861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:21:19.710587 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.710569 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 15:21:19.720697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.720676 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 15:21:19.731182 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.731154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rll6h\" (UniqueName: \"kubernetes.io/projected/5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf-kube-api-access-rll6h\") pod \"network-check-target-l9fw9\" (UID: \"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf\") " pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:21:19.840064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.840024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bw5ll\"" Jul 12 15:21:19.845118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.845101 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h7cqj" Jul 12 15:21:19.847603 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.847587 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:21:19.981457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.981272 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h7cqj"] Jul 12 15:21:19.983756 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:21:19.983732 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1bf5087_3ddd_4030_bdca_c5299683ad1d.slice/crio-5d4d3213a41da3864b2f0714930eb5f7083225cdac781f85384b2a7bbeb552a7 WatchSource:0}: Error finding container 5d4d3213a41da3864b2f0714930eb5f7083225cdac781f85384b2a7bbeb552a7: Status 404 returned error can't find the container with id 5d4d3213a41da3864b2f0714930eb5f7083225cdac781f85384b2a7bbeb552a7 Jul 12 15:21:19.996713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:19.996687 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9fw9"] Jul 12 15:21:19.999612 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:21:19.999585 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7a0752_aa95_4a7e_9ab0_d8bb1824bbdf.slice/crio-897a3e77045a8b7f26da700466d28a1c600aa85623ae994c42e92deac0902344 WatchSource:0}: Error finding container 897a3e77045a8b7f26da700466d28a1c600aa85623ae994c42e92deac0902344: Status 404 returned error can't find the container with id 897a3e77045a8b7f26da700466d28a1c600aa85623ae994c42e92deac0902344 Jul 12 15:21:20.312523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:20.312484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9fw9" event={"ID":"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf","Type":"ContainerStarted","Data":"897a3e77045a8b7f26da700466d28a1c600aa85623ae994c42e92deac0902344"} Jul 12 15:21:20.313488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:20.313465 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h7cqj" event={"ID":"a1bf5087-3ddd-4030-bdca-c5299683ad1d","Type":"ContainerStarted","Data":"5d4d3213a41da3864b2f0714930eb5f7083225cdac781f85384b2a7bbeb552a7"} Jul 12 15:21:22.228516 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:22.228479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:22.228563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:22.228605 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228651 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228708 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228712 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228724 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:54.2287004 +0000 UTC m=+99.778901168 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228729 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228763 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:54.228746764 +0000 UTC m=+99.778947516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:21:22.228932 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:22.228780 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:54.228771054 +0000 UTC m=+99.778971808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:21:24.200370 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:24.200334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:24.257459 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:24.257415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:24.320785 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:24.320754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:24.421803 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:24.421760 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:24.475645 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:24.475562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:25.332046 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:25.332012 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9fw9" event={"ID":"5b7a0752-aa95-4a7e-9ab0-d8bb1824bbdf","Type":"ContainerStarted","Data":"5c6a98edc638421b0e3583203de8d79c59ca83fc2bc60a46aa84b4e733f4d668"} Jul 12 15:21:25.332474 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:25.332144 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:21:25.333523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:25.333500 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h7cqj" event={"ID":"a1bf5087-3ddd-4030-bdca-c5299683ad1d","Type":"ContainerStarted","Data":"ff5b2715662d030c0e7fb615e8686e02c4c87ff97b08a7b447de06940f54132d"} Jul 12 15:21:25.346903 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:25.346860 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l9fw9" podStartSLOduration=66.574976675 podStartE2EDuration="1m11.346845958s" totalImagesPullingTime="4.771869283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:25.345976896 +0000 UTC m=+70.896177671" watchObservedRunningTime="2026-07-12 15:21:25.346845958 +0000 UTC m=+70.897046733" Jul 12 15:21:25.367100 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:25.367013 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h7cqj" podStartSLOduration=65.574042392 podStartE2EDuration="1m10.367001746s" totalImagesPullingTime="4.792959354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:25.366706858 +0000 UTC m=+70.916907645" watchObservedRunningTime="2026-07-12 15:21:25.367001746 +0000 UTC m=+70.917202527" Jul 12 15:21:30.492082 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:30.492038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:30.592948 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:30.592916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:53.664387 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:53.664262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:21:54.275999 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:54.275952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:21:54.276177 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:54.276021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:21:54.276177 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:54.276061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:21:54.276177 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276136 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:21:54.276177 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276152 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:54.276308 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276175 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:21:54.276308 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276193 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:21:54.276308 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276234 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:22:58.27621037 +0000 UTC m=+163.826411124 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:21:54.276308 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276257 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:22:58.276243043 +0000 UTC m=+163.826443797 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:21:54.276308 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:21:54.276275 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:22:58.27626561 +0000 UTC m=+163.826466366 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:21:56.338046 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:21:56.338018 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l9fw9" Jul 12 15:22:23.689266 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:23.689224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:22:23.689796 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:23.689357 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:22:23.689796 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:23.689447 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:25.689431429 +0000 UTC m=+251.239632183 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : secret "metrics-daemon-secret" not found Jul 12 15:22:53.345484 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:53.345440 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" podUID="4ea66087-0992-41de-87d9-59515e7eb1d6" Jul 12 15:22:53.360631 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:53.360595 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" Jul 12 15:22:53.412942 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:53.412898 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ckjzp" podUID="f0733aa5-675c-41c8-b924-87c2b65fb004" Jul 12 15:22:53.589944 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:53.589916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:22:53.590107 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:53.589915 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:22:53.590107 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:53.589923 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:22:54.951538 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:54.951500 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:22:58.361167 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:58.361127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:22:58.361167 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:58.361173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361286 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361304 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:22:58.361317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361326 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58c57d8c-bp8c5: secret "image-registry-tls" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361331 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls podName:f0733aa5-675c-41c8-b924-87c2b65fb004 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.361318383 +0000 UTC m=+285.911519137 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls") pod "dns-default-ckjzp" (UID: "f0733aa5-675c-41c8-b924-87c2b65fb004") : secret "dns-default-metrics-tls" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361445 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls podName:723e2194-9110-4c01-a873-c02184ea3cc0 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.361388569 +0000 UTC m=+285.911589322 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls") pod "image-registry-6d58c57d8c-bp8c5" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0") : secret "image-registry-tls" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361450 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 15:22:58.361707 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:22:58.361515 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert podName:4ea66087-0992-41de-87d9-59515e7eb1d6 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.361502844 +0000 UTC m=+285.911703597 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-2gmjr" (UID: "4ea66087-0992-41de-87d9-59515e7eb1d6") : secret "networking-console-plugin-cert" not found Jul 12 15:23:00.609292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.609258 2563 generic.go:356] "Generic (PLEG): container finished" podID="efb84714-145d-4faa-b47d-87a8c2691518" containerID="5413187def2ec5ede7b254003dbe31f06818092068249dcdc02daf380083b92c" exitCode=255 Jul 12 15:23:00.609705 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.609333 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerDied","Data":"5413187def2ec5ede7b254003dbe31f06818092068249dcdc02daf380083b92c"} Jul 12 15:23:00.609767 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.609709 2563 scope.go:117] "RemoveContainer" containerID="5413187def2ec5ede7b254003dbe31f06818092068249dcdc02daf380083b92c" Jul 12 15:23:00.610770 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.610747 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9db2933-abb6-488a-b9c4-15daa3a4c734" containerID="290fa4996ceef10e6e5d4890796bd71fea85a1500277f68d45254c0892ea2cdd" exitCode=1 Jul 12 15:23:00.610844 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.610780 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerDied","Data":"290fa4996ceef10e6e5d4890796bd71fea85a1500277f68d45254c0892ea2cdd"} Jul 12 15:23:00.611089 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.611074 2563 scope.go:117] "RemoveContainer" containerID="290fa4996ceef10e6e5d4890796bd71fea85a1500277f68d45254c0892ea2cdd" Jul 12 15:23:00.678775 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.678744 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:23:00.686103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:00.686070 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:23:01.251891 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:01.251841 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:23:01.614923 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:01.614836 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerStarted","Data":"ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe"} Jul 12 15:23:01.615390 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:01.615020 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:23:01.616705 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:01.616685 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:23:01.616822 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:01.616804 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerStarted","Data":"832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32"} Jul 12 15:23:05.924497 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:23:05.924381 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:24:17.231801 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:17.231749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:17.314259 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:17.314219 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:19.682278 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:19.682232 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:19.733921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:19.733881 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:21.865067 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.865031 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9r594"] Jul 12 15:24:21.868137 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.868114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:21.873112 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.873082 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 15:24:21.873454 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.873430 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 15:24:21.873529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.873491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 15:24:21.873571 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.873491 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 15:24:21.874506 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.874320 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4k8zs\"" Jul 12 15:24:21.883644 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.883605 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9r594"] Jul 12 15:24:21.911669 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.911644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:21.961783 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.961744 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:21.981016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.980988 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1142c0f9-4f1f-449a-a1c1-e109015a7e53-data-volume\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:21.981093 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.981057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xwz\" (UniqueName: \"kubernetes.io/projected/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-api-access-v4xwz\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:21.981093 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.981088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:21.981157 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.981121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1142c0f9-4f1f-449a-a1c1-e109015a7e53-crio-socket\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:21.981201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:21.981183 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.073236 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.073196 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.082311 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1142c0f9-4f1f-449a-a1c1-e109015a7e53-data-volume\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082384 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xwz\" (UniqueName: \"kubernetes.io/projected/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-api-access-v4xwz\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082455 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082507 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.082492 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:22.082570 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.082559 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:22.582540386 +0000 UTC m=+248.132741144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : configmap "kube-rbac-proxy" not found Jul 12 15:24:22.082624 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1142c0f9-4f1f-449a-a1c1-e109015a7e53-crio-socket\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082660 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082724 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082692 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1142c0f9-4f1f-449a-a1c1-e109015a7e53-data-volume\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082724 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.082714 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1142c0f9-4f1f-449a-a1c1-e109015a7e53-crio-socket\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.082812 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.082775 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.082849 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.082827 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:22.582815041 +0000 UTC m=+248.133015794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.098825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.098801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xwz\" (UniqueName: \"kubernetes.io/projected/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-api-access-v4xwz\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.162598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.162565 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.262478 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.262446 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.416875 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.416799 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.467576 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.467543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.588123 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.588093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.588296 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.588144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:22.588296 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.588231 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:22.588296 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.588255 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.588296 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.588295 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:23.58828326 +0000 UTC m=+249.138484017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.588469 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:22.588307 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:23.58830181 +0000 UTC m=+249.138502563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : configmap "kube-rbac-proxy" not found Jul 12 15:24:22.702439 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.702341 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:22.803746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:22.803717 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:23.595176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:23.595126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:23.595631 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:23.595194 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:23.595631 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:23.595270 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:23.595631 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:23.595330 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:23.595631 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:23.595341 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:25.595324343 +0000 UTC m=+251.145525097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : configmap "kube-rbac-proxy" not found Jul 12 15:24:23.595631 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:23.595386 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:25.595373247 +0000 UTC m=+251.145574004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:24.050447 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:24.050417 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-77n27_ca059fb5-eee1-4767-bb92-d39f488b76a2/dns-node-resolver/0.log" Jul 12 15:24:24.325785 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:24.325704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:24.379979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:24.379952 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:24.694512 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:24.694483 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:24.849646 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:24.849619 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b6qnh_502c5db6-ee40-4043-84d1-b582261cc920/node-ca/0.log" Jul 12 15:24:25.611010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:25.610973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:25.611198 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:25.611038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:25.611198 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.611091 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:25.611198 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.611126 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:25.611198 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.611163 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:29.611148209 +0000 UTC m=+255.161348965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : configmap "kube-rbac-proxy" not found Jul 12 15:24:25.611198 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.611184 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:29.611172211 +0000 UTC m=+255.161372964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:25.648887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:25.648861 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgtjd_9f686a4e-6106-4ce9-8438-fefae0571b84/serve-healthcheck-canary/0.log" Jul 12 15:24:25.712280 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:25.712247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:24:25.712652 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.712432 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:24:25.712652 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:25.712497 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs podName:733d8099-0f3f-4195-a5fd-4813d320ce42 nodeName:}" failed. No retries permitted until 2026-07-12 15:26:27.712480683 +0000 UTC m=+373.262681437 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs") pod "network-metrics-daemon-dtq8d" (UID: "733d8099-0f3f-4195-a5fd-4813d320ce42") : secret "metrics-daemon-secret" not found Jul 12 15:24:25.793176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:25.793141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:27.608366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:27.608279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:27.841417 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:27.841365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:27.942080 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:27.942043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:28.043140 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:28.043110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:28.145782 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:28.145753 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:28.247980 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:28.247886 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:28.705200 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:28.705140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:28.902069 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:28.902035 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:29.299903 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:29.299864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:29.644468 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:29.644366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:29.644612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:29.644484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:29.644612 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:29.644515 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:29.644612 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:29.644564 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:29.644612 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:29.644576 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:37.64456227 +0000 UTC m=+263.194763022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:29.644612 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:29.644604 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm podName:1142c0f9-4f1f-449a-a1c1-e109015a7e53 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:37.644593095 +0000 UTC m=+263.194793848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9r594" (UID: "1142c0f9-4f1f-449a-a1c1-e109015a7e53") : configmap "kube-rbac-proxy" not found Jul 12 15:24:29.898015 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:29.897933 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:30.500633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:30.500599 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:30.702996 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:30.702965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:30.905744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:30.905704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:31.100321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:31.100285 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:31.301183 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:31.301103 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:31.498152 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:31.498122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:31.770900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:31.770869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:32.102770 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:32.102687 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:32.500932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:32.500904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:32.696829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:32.696799 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:32.898743 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:32.898659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:33.104960 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:33.104932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:33.901493 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:33.901462 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:34.106540 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:34.106504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:34.499139 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:34.499109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:34.700186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:34.700144 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:34.906334 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:34.906305 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:35.111973 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:35.111946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:35.301685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:35.301601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:35.698685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:35.698650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:36.496976 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:36.496944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:36.701334 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:36.701306 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:37.104890 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.104853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:37.301693 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.299539 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:37.707505 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.707466 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:37.707866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.707517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:37.707970 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.707951 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1142c0f9-4f1f-449a-a1c1-e109015a7e53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:37.709933 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.709914 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1142c0f9-4f1f-449a-a1c1-e109015a7e53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9r594\" (UID: \"1142c0f9-4f1f-449a-a1c1-e109015a7e53\") " pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:37.777023 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.776986 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9r594" Jul 12 15:24:37.901418 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.901376 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9r594"] Jul 12 15:24:37.903948 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:37.903918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:37.904085 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:24:37.904005 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1142c0f9_4f1f_449a_a1c1_e109015a7e53.slice/crio-256f92b641236118d5bf276ceb9dd942237ecea317c9aa9e8c9b14e8670f16cb WatchSource:0}: Error finding container 256f92b641236118d5bf276ceb9dd942237ecea317c9aa9e8c9b14e8670f16cb: Status 404 returned error can't find the container with id 256f92b641236118d5bf276ceb9dd942237ecea317c9aa9e8c9b14e8670f16cb Jul 12 15:24:38.099841 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:38.099755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:38.300411 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:38.300366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:38.899118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:38.899081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9r594" event={"ID":"1142c0f9-4f1f-449a-a1c1-e109015a7e53","Type":"ContainerStarted","Data":"256f92b641236118d5bf276ceb9dd942237ecea317c9aa9e8c9b14e8670f16cb"} Jul 12 15:24:41.911036 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:41.911003 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9r594" event={"ID":"1142c0f9-4f1f-449a-a1c1-e109015a7e53","Type":"ContainerStarted","Data":"fd0ab287006422350842624d96cc962944afebeb441ca90440e269529a8d1d90"} Jul 12 15:24:42.384907 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:42.384875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:42.486367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:42.486272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:42.904316 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:42.904280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:42.915924 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:42.915885 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9r594" event={"ID":"1142c0f9-4f1f-449a-a1c1-e109015a7e53","Type":"ContainerStarted","Data":"737733b1ffd9a81fd8a8b36fbbc2cdbb76daba32a9e4ff4d4ddc81a95f1a647a"} Jul 12 15:24:43.063085 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.063046 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:43.117825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.117791 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:43.464741 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.464707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:43.512774 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.512741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:43.688030 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.687997 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:43.789226 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:43.789135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.001683 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.001647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.101285 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.101192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.274170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.274135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.475306 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.475266 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.612381 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.612350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:44.931004 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.930969 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9r594" event={"ID":"1142c0f9-4f1f-449a-a1c1-e109015a7e53","Type":"ContainerStarted","Data":"0376d2229a939ce59e3830e2933756331ade7cd952c1c10624119b220c6f20d6"} Jul 12 15:24:44.949732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:44.949685 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9r594" podStartSLOduration=17.261567978 podStartE2EDuration="23.949669834s" totalImagesPullingTime="6.688101856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:24:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:44.949437879 +0000 UTC m=+270.499638673" watchObservedRunningTime="2026-07-12 15:24:44.949669834 +0000 UTC m=+270.499870608" Jul 12 15:24:45.140998 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:45.140915 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:45.242472 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:45.242438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:45.340265 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:45.340229 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:50.694658 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:50.694611 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" podUID="9f5c2f9f-c928-400a-b6f9-15e663c2e9c2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 15:24:51.263190 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:51.263153 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:51.363731 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:51.363700 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:51.429250 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:51.429206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:51.796704 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:51.796672 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:55.451527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:55.451495 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:55.512639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:55.512605 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:55.639159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:55.639120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:56.591164 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:56.591125 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" Jul 12 15:24:56.591164 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:56.591150 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ckjzp" podUID="f0733aa5-675c-41c8-b924-87c2b65fb004" Jul 12 15:24:56.591164 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:24:56.591128 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" podUID="4ea66087-0992-41de-87d9-59515e7eb1d6" Jul 12 15:24:56.958511 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:56.958481 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:24:56.958696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:56.958480 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:24:56.958696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:56.958481 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:24:57.562606 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:57.562577 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:58.315657 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:58.315623 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:58.405870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:58.405834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:58.895634 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:58.895603 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:58.946614 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:58.946569 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:59.593976 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.593938 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:59.796883 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.796853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:59.846982 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.846912 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-66ssc"] Jul 12 15:24:59.850345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.850325 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.852642 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.852621 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 15:24:59.852904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.852875 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 15:24:59.853032 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.852905 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 15:24:59.853032 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.852918 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 15:24:59.853032 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.852965 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p4dw4\"" Jul 12 15:24:59.853804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.853791 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 15:24:59.853970 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.853958 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 15:24:59.876818 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.876788 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:59.934476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.934445 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:24:59.984331 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8n2v\" (UniqueName: \"kubernetes.io/projected/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-kube-api-access-b8n2v\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984331 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984350 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984378 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-wtmp\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-root\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-sys\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-textfile\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-metrics-client-ca\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:24:59.984568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:24:59.984497 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-accelerators-collector-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8n2v\" (UniqueName: \"kubernetes.io/projected/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-kube-api-access-b8n2v\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085666 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-wtmp\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-root\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-sys\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.085811 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:00.085775 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:00.085862 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls podName:197f34d3-2a2e-4ab1-8f9a-e5665bef9360 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.585841404 +0000 UTC m=+286.136042190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls") pod "node-exporter-66ssc" (UID: "197f34d3-2a2e-4ab1-8f9a-e5665bef9360") : secret "node-exporter-tls" not found Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085866 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-sys\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085865 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-root\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085783 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-textfile\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-metrics-client-ca\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085873 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-wtmp\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.085975 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-accelerators-collector-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.086152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-textfile\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086410 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.086379 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-metrics-client-ca\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.086476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.086455 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-accelerators-collector-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.087952 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.087935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.092835 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.092812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.094219 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.094200 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8n2v\" (UniqueName: \"kubernetes.io/projected/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-kube-api-access-b8n2v\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.193403 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.193373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.293701 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.293670 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.389344 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.389309 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:25:00.389555 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.389431 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:25:00.389555 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.389497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:00.391800 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.391773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0733aa5-675c-41c8-b924-87c2b65fb004-metrics-tls\") pod \"dns-default-ckjzp\" (UID: \"f0733aa5-675c-41c8-b924-87c2b65fb004\") " pod="openshift-dns/dns-default-ckjzp" Jul 12 15:25:00.391906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.391860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ea66087-0992-41de-87d9-59515e7eb1d6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-2gmjr\" (UID: \"4ea66087-0992-41de-87d9-59515e7eb1d6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:25:00.391906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.391892 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"image-registry-6d58c57d8c-bp8c5\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:00.394863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.394843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.561518 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.561439 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ns4rd\"" Jul 12 15:25:00.561518 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.561487 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qx8d5\"" Jul 12 15:25:00.561704 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.561691 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wn7g5\"" Jul 12 15:25:00.569668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.569651 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" Jul 12 15:25:00.569722 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.569676 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:00.569768 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.569755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:25:00.590682 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.590653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.593036 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.593016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/197f34d3-2a2e-4ab1-8f9a-e5665bef9360-node-exporter-tls\") pod \"node-exporter-66ssc\" (UID: \"197f34d3-2a2e-4ab1-8f9a-e5665bef9360\") " pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.598248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.598224 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.694685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.694518 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" podUID="9f5c2f9f-c928-400a-b6f9-15e663c2e9c2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 15:25:00.715216 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.715183 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-2gmjr"] Jul 12 15:25:00.719423 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:25:00.719377 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea66087_0992_41de_87d9_59515e7eb1d6.slice/crio-d220c5ba131bb91e10180b4e461d1a7e3b7e84bbd996c959f7a89def2bb82734 WatchSource:0}: Error finding container d220c5ba131bb91e10180b4e461d1a7e3b7e84bbd996c959f7a89def2bb82734: Status 404 returned error can't find the container with id d220c5ba131bb91e10180b4e461d1a7e3b7e84bbd996c959f7a89def2bb82734 Jul 12 15:25:00.759194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.759168 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66ssc" Jul 12 15:25:00.810682 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.810636 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.819312 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:25:00.819240 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197f34d3_2a2e_4ab1_8f9a_e5665bef9360.slice/crio-10ef3855dbaf70ec814701923327b86965b99068b67edae78e527429e891cf03 WatchSource:0}: Error finding container 10ef3855dbaf70ec814701923327b86965b99068b67edae78e527429e891cf03: Status 404 returned error can't find the container with id 10ef3855dbaf70ec814701923327b86965b99068b67edae78e527429e891cf03 Jul 12 15:25:00.883172 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.883137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:00.948439 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.948391 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ckjzp"] Jul 12 15:25:00.951449 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.951423 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:25:00.952082 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:25:00.952060 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0733aa5_675c_41c8_b924_87c2b65fb004.slice/crio-d75c49f5c6f87dd215d476f254ff00747bde4ed1baa1b31263be6fc38d09c89d WatchSource:0}: Error finding container d75c49f5c6f87dd215d476f254ff00747bde4ed1baa1b31263be6fc38d09c89d: Status 404 returned error can't find the container with id d75c49f5c6f87dd215d476f254ff00747bde4ed1baa1b31263be6fc38d09c89d Jul 12 15:25:00.954025 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:25:00.954005 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723e2194_9110_4c01_a873_c02184ea3cc0.slice/crio-fefca756514f339048bd5a7216e373a7202b9309a719af1ca89e5da360282093 WatchSource:0}: Error finding container fefca756514f339048bd5a7216e373a7202b9309a719af1ca89e5da360282093: Status 404 returned error can't find the container with id fefca756514f339048bd5a7216e373a7202b9309a719af1ca89e5da360282093 Jul 12 15:25:00.976904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.976882 2563 generic.go:356] "Generic (PLEG): container finished" podID="efb84714-145d-4faa-b47d-87a8c2691518" containerID="832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32" exitCode=255 Jul 12 15:25:00.977007 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.976955 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerDied","Data":"832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32"} Jul 12 15:25:00.977064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.977004 2563 scope.go:117] "RemoveContainer" containerID="5413187def2ec5ede7b254003dbe31f06818092068249dcdc02daf380083b92c" Jul 12 15:25:00.977354 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.977322 2563 scope.go:117] "RemoveContainer" containerID="832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32" Jul 12 15:25:00.977523 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:00.977503 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7fbb6589d4-mp252_open-cluster-management-agent-addon(efb84714-145d-4faa-b47d-87a8c2691518)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" podUID="efb84714-145d-4faa-b47d-87a8c2691518" Jul 12 15:25:00.978111 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.978088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66ssc" event={"ID":"197f34d3-2a2e-4ab1-8f9a-e5665bef9360","Type":"ContainerStarted","Data":"10ef3855dbaf70ec814701923327b86965b99068b67edae78e527429e891cf03"} Jul 12 15:25:00.979059 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.979042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjzp" event={"ID":"f0733aa5-675c-41c8-b924-87c2b65fb004","Type":"ContainerStarted","Data":"d75c49f5c6f87dd215d476f254ff00747bde4ed1baa1b31263be6fc38d09c89d"} Jul 12 15:25:00.980026 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.980000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" event={"ID":"723e2194-9110-4c01-a873-c02184ea3cc0","Type":"ContainerStarted","Data":"fefca756514f339048bd5a7216e373a7202b9309a719af1ca89e5da360282093"} Jul 12 15:25:00.981064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.981048 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" event={"ID":"4ea66087-0992-41de-87d9-59515e7eb1d6","Type":"ContainerStarted","Data":"d220c5ba131bb91e10180b4e461d1a7e3b7e84bbd996c959f7a89def2bb82734"} Jul 12 15:25:00.982961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.982661 2563 generic.go:356] "Generic (PLEG): container finished" podID="e9db2933-abb6-488a-b9c4-15daa3a4c734" containerID="ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe" exitCode=1 Jul 12 15:25:00.982961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.982715 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerDied","Data":"ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe"} Jul 12 15:25:00.983085 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.983030 2563 scope.go:117] "RemoveContainer" containerID="ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe" Jul 12 15:25:00.983211 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:00.983186 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-6c59565cf5-hd4vq_open-cluster-management-agent-addon(e9db2933-abb6-488a-b9c4-15daa3a4c734)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" podUID="e9db2933-abb6-488a-b9c4-15daa3a4c734" Jul 12 15:25:00.989179 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:00.989163 2563 scope.go:117] "RemoveContainer" containerID="290fa4996ceef10e6e5d4890796bd71fea85a1500277f68d45254c0892ea2cdd" Jul 12 15:25:01.178329 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.178290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:01.431226 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.431197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:01.615080 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.615037 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:25:01.732059 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.731960 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:01.989416 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.989324 2563 scope.go:117] "RemoveContainer" containerID="ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe" Jul 12 15:25:01.989585 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:01.989558 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-6c59565cf5-hd4vq_open-cluster-management-agent-addon(e9db2933-abb6-488a-b9c4-15daa3a4c734)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" podUID="e9db2933-abb6-488a-b9c4-15daa3a4c734" Jul 12 15:25:01.993538 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.993466 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" event={"ID":"723e2194-9110-4c01-a873-c02184ea3cc0","Type":"ContainerStarted","Data":"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2"} Jul 12 15:25:01.993695 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:01.993619 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:02.027459 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.026267 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" podStartSLOduration=287.026245911 podStartE2EDuration="4m47.026245911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:02.025751696 +0000 UTC m=+287.575952499" watchObservedRunningTime="2026-07-12 15:25:02.026245911 +0000 UTC m=+287.576446686" Jul 12 15:25:02.736391 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.736350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:02.839928 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.839863 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:02.998064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.998033 2563 generic.go:356] "Generic (PLEG): container finished" podID="197f34d3-2a2e-4ab1-8f9a-e5665bef9360" containerID="f91d9a3447f89c1b7560f5e4fdaaf4171cbf2bdd17548f78a1b2f1d2e302e01f" exitCode=0 Jul 12 15:25:02.998256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.998135 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66ssc" event={"ID":"197f34d3-2a2e-4ab1-8f9a-e5665bef9360","Type":"ContainerDied","Data":"f91d9a3447f89c1b7560f5e4fdaaf4171cbf2bdd17548f78a1b2f1d2e302e01f"} Jul 12 15:25:02.999657 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:02.999636 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" event={"ID":"4ea66087-0992-41de-87d9-59515e7eb1d6","Type":"ContainerStarted","Data":"f18940a944bb2d3aed0de225ade7f96a62572f6c3dd83d91f93d2599c2a4af6e"} Jul 12 15:25:03.023807 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:03.023728 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:03.030709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:03.030669 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-2gmjr" podStartSLOduration=286.521276477 podStartE2EDuration="4m48.030655905s" totalImagesPullingTime="1.509379428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:03.030451966 +0000 UTC m=+288.580652765" watchObservedRunningTime="2026-07-12 15:25:03.030655905 +0000 UTC m=+288.580856676" Jul 12 15:25:03.130386 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:03.130363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:03.215937 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:03.215882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:04.004130 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.004091 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjzp" event={"ID":"f0733aa5-675c-41c8-b924-87c2b65fb004","Type":"ContainerStarted","Data":"bb5fc582575a3e82cccb2fe35c609d8fa6dacbbe775ce216aadfee65604bbd38"} Jul 12 15:25:04.004600 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.004187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ckjzp" event={"ID":"f0733aa5-675c-41c8-b924-87c2b65fb004","Type":"ContainerStarted","Data":"6545b5cc0200939795fbedf718546968647796735a5f36c3c5bb833bda49ced8"} Jul 12 15:25:04.004600 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.004219 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:25:04.006058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.006036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66ssc" event={"ID":"197f34d3-2a2e-4ab1-8f9a-e5665bef9360","Type":"ContainerStarted","Data":"a1e955ce6beff9b2648bbfca3050d7313fad03c1fc01e2319581e8a1fead8c79"} Jul 12 15:25:04.006166 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.006077 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66ssc" event={"ID":"197f34d3-2a2e-4ab1-8f9a-e5665bef9360","Type":"ContainerStarted","Data":"25b6c39fde638d255fb0d3badae3982fadedcc4ff5e012ad460a993671fce693"} Jul 12 15:25:04.022171 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.022121 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ckjzp" podStartSLOduration=251.984477012 podStartE2EDuration="4m14.022104185s" totalImagesPullingTime="2.037627173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:04.020238119 +0000 UTC m=+289.570438893" watchObservedRunningTime="2026-07-12 15:25:04.022104185 +0000 UTC m=+289.572304959" Jul 12 15:25:04.037199 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.037150 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-66ssc" podStartSLOduration=3.555741882 podStartE2EDuration="5.037134078s" totalImagesPullingTime="1.408134154s" totalInitContainerRuntime="73.258042ms" isStatefulPod=true podCreationTimestamp="2026-07-12 15:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:04.036323321 +0000 UTC m=+289.586524115" watchObservedRunningTime="2026-07-12 15:25:04.037134078 +0000 UTC m=+289.587334853" Jul 12 15:25:04.621414 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.621364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:04.677771 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:04.677706 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:05.033945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:05.033906 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:05.973902 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:05.973832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:08.925492 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:08.925452 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dtq8d" podUID="733d8099-0f3f-4195-a5fd-4813d320ce42" Jul 12 15:25:09.262279 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:09.262191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:10.679523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.679473 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" Jul 12 15:25:10.679916 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.679836 2563 scope.go:117] "RemoveContainer" containerID="832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32" Jul 12 15:25:10.680000 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:10.679981 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7fbb6589d4-mp252_open-cluster-management-agent-addon(efb84714-145d-4faa-b47d-87a8c2691518)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" podUID="efb84714-145d-4faa-b47d-87a8c2691518" Jul 12 15:25:10.686262 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.686241 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:25:10.686610 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.686597 2563 scope.go:117] "RemoveContainer" containerID="ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe" Jul 12 15:25:10.686745 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:10.686729 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-6c59565cf5-hd4vq_open-cluster-management-agent-addon(e9db2933-abb6-488a-b9c4-15daa3a4c734)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" podUID="e9db2933-abb6-488a-b9c4-15daa3a4c734" Jul 12 15:25:10.693953 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.693922 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" podUID="9f5c2f9f-c928-400a-b6f9-15e663c2e9c2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 12 15:25:10.694080 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.693990 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" Jul 12 15:25:10.694441 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.694414 2563 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ae3274f56044f36af9eadaeb877257281776075f26a70fb348b9091cbaafb45c"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 12 15:25:10.694490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:10.694462 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" podUID="9f5c2f9f-c928-400a-b6f9-15e663c2e9c2" containerName="service-proxy" containerID="cri-o://ae3274f56044f36af9eadaeb877257281776075f26a70fb348b9091cbaafb45c" gracePeriod=30 Jul 12 15:25:11.031800 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:11.031705 2563 generic.go:356] "Generic (PLEG): container finished" podID="9f5c2f9f-c928-400a-b6f9-15e663c2e9c2" containerID="ae3274f56044f36af9eadaeb877257281776075f26a70fb348b9091cbaafb45c" exitCode=2 Jul 12 15:25:11.031800 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:11.031782 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerDied","Data":"ae3274f56044f36af9eadaeb877257281776075f26a70fb348b9091cbaafb45c"} Jul 12 15:25:11.031973 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:11.031830 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56ff8f58bb-lnwwh" event={"ID":"9f5c2f9f-c928-400a-b6f9-15e663c2e9c2","Type":"ContainerStarted","Data":"2ce4492e95a2d8ed563102df49a383c0839628e687c0ac42343c3c1977970356"} Jul 12 15:25:14.011898 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:14.011870 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ckjzp" Jul 12 15:25:14.790760 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:14.790727 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:25:14.791986 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:14.791954 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:25:14.797696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:14.797675 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 15:25:16.723559 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.723523 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:25:16.797675 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.727845 2563 patch_prober.go:28] interesting pod/image-registry-6d58c57d8c-bp8c5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 12 15:25:16.797675 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.727896 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 15:25:16.798841 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.798814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:16.877545 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.877518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:16.937310 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:16.937280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:17.039344 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:17.039261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:17.139267 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:17.139240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:18.018552 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:18.018520 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:18.118069 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:18.117999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:18.217174 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:18.217144 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:18.274465 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:18.274367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:20.169201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:20.169167 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:20.223382 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:20.223349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:22.928744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:22.928722 2563 scope.go:117] "RemoveContainer" containerID="832c05ac574f21ae36b3cbf55b2411ccb2c8d7fd7d65afc0ffe28a18740edd32" Jul 12 15:25:23.079285 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:23.079255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fbb6589d4-mp252" event={"ID":"efb84714-145d-4faa-b47d-87a8c2691518","Type":"ContainerStarted","Data":"76b02364e75f957cfb5383668a62ba6eb0c756a0d564a7ff02c5b353d859f84a"} Jul 12 15:25:23.137322 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:23.137288 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:23.238224 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:23.238148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:23.924473 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:23.924427 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:25:23.924704 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:23.924686 2563 scope.go:117] "RemoveContainer" containerID="ca3da8d5e7307a84a9dd5021c9b0131cc6ac1fb4150676b89a853e30dc5847fe" Jul 12 15:25:24.084251 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:24.084222 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" event={"ID":"e9db2933-abb6-488a-b9c4-15daa3a4c734","Type":"ContainerStarted","Data":"3239d84dfad7694375a1140d0ed71aaa97d3d5790de44d150a88fd926486ca23"} Jul 12 15:25:24.084660 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:24.084504 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:25:24.086088 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:24.086072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c59565cf5-hd4vq" Jul 12 15:25:26.728130 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:26.728101 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:27.292242 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:27.292206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:27.345436 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:27.345380 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:28.992091 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:28.992060 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:29.049024 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:29.048999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:32.138628 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:32.138600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:32.197518 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:32.197486 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:32.495231 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:32.495206 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgtjd_9f686a4e-6106-4ce9-8438-fefae0571b84/serve-healthcheck-canary/0.log" Jul 12 15:25:41.149777 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:41.149741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:41.744221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:41.744160 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" containerName="registry" containerID="cri-o://4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2" gracePeriod=30 Jul 12 15:25:41.979527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:41.979506 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:42.009170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009096 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009128 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009148 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009165 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009513 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009189 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009513 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009314 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009513 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009347 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.009639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009613 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.009778 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.009757 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tl6r\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r\") pod \"723e2194-9110-4c01-a873-c02184ea3cc0\" (UID: \"723e2194-9110-4c01-a873-c02184ea3cc0\") " Jul 12 15:25:42.010315 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.010055 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-registry-certificates\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.010315 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.010082 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.012272 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.012203 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:42.012384 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.012287 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:42.012384 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.012375 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:25:42.012546 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.012524 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r" (OuterVolumeSpecName: "kube-api-access-6tl6r") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "kube-api-access-6tl6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:25:42.013009 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.012978 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:25:42.020550 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.020526 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "723e2194-9110-4c01-a873-c02184ea3cc0" (UID: "723e2194-9110-4c01-a873-c02184ea3cc0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:25:42.110688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110649 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-registry-tls\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110680 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-installation-pull-secrets\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110691 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-bound-sa-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110700 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/723e2194-9110-4c01-a873-c02184ea3cc0-ca-trust-extracted\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110934 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110710 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/723e2194-9110-4c01-a873-c02184ea3cc0-image-registry-private-configuration\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110934 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110719 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/723e2194-9110-4c01-a873-c02184ea3cc0-trusted-ca\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.110934 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.110727 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tl6r\" (UniqueName: \"kubernetes.io/projected/723e2194-9110-4c01-a873-c02184ea3cc0-kube-api-access-6tl6r\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.145004 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.144971 2563 generic.go:356] "Generic (PLEG): container finished" podID="723e2194-9110-4c01-a873-c02184ea3cc0" containerID="4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2" exitCode=0 Jul 12 15:25:42.145135 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.145015 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" Jul 12 15:25:42.145135 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.145065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" event={"ID":"723e2194-9110-4c01-a873-c02184ea3cc0","Type":"ContainerDied","Data":"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2"} Jul 12 15:25:42.145135 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.145132 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58c57d8c-bp8c5" event={"ID":"723e2194-9110-4c01-a873-c02184ea3cc0","Type":"ContainerDied","Data":"fefca756514f339048bd5a7216e373a7202b9309a719af1ca89e5da360282093"} Jul 12 15:25:42.145236 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.145151 2563 scope.go:117] "RemoveContainer" containerID="4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2" Jul 12 15:25:42.153345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.153330 2563 scope.go:117] "RemoveContainer" containerID="4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2" Jul 12 15:25:42.153645 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:25:42.153599 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2\": container with ID starting with 4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2 not found: ID does not exist" containerID="4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2" Jul 12 15:25:42.153645 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.153629 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2"} err="failed to get container status \"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2\": rpc error: code = NotFound desc = could not find container \"4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2\": container with ID starting with 4ed16c6864a02e61a457a199dfd66aea8c4a13642225a7a53190c424d5f13fd2 not found: ID does not exist" Jul 12 15:25:42.164115 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.164060 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:25:42.167977 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.167958 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d58c57d8c-bp8c5"] Jul 12 15:25:42.930146 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:42.930111 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" path="/var/lib/kubelet/pods/723e2194-9110-4c01-a873-c02184ea3cc0/volumes" Jul 12 15:25:43.990854 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:43.990822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:25:44.057851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:25:44.057801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:04.046452 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:04.046419 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:04.144076 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:04.144048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:04.197278 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:04.197240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:18.979128 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:18.979092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:19.230916 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:19.230823 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:27.742763 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.742722 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:26:27.745183 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.745164 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/733d8099-0f3f-4195-a5fd-4813d320ce42-metrics-certs\") pod \"network-metrics-daemon-dtq8d\" (UID: \"733d8099-0f3f-4195-a5fd-4813d320ce42\") " pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:26:27.822860 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.822828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:27.828171 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.828151 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ztf2\"" Jul 12 15:26:27.836120 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.836104 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtq8d" Jul 12 15:26:27.871130 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.871098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:27.986552 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.986522 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dtq8d"] Jul 12 15:26:27.989477 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:26:27.989454 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733d8099_0f3f_4195_a5fd_4813d320ce42.slice/crio-b6f798c28e5cfc33bd79f3361e37e76a8bf621c02ad6fd89e067d6610244f7ae WatchSource:0}: Error finding container b6f798c28e5cfc33bd79f3361e37e76a8bf621c02ad6fd89e067d6610244f7ae: Status 404 returned error can't find the container with id b6f798c28e5cfc33bd79f3361e37e76a8bf621c02ad6fd89e067d6610244f7ae Jul 12 15:26:27.991368 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:27.991353 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:26:28.298662 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:28.298578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtq8d" event={"ID":"733d8099-0f3f-4195-a5fd-4813d320ce42","Type":"ContainerStarted","Data":"b6f798c28e5cfc33bd79f3361e37e76a8bf621c02ad6fd89e067d6610244f7ae"} Jul 12 15:26:29.303921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:29.303893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtq8d" event={"ID":"733d8099-0f3f-4195-a5fd-4813d320ce42","Type":"ContainerStarted","Data":"faad9b919673b4565c667cff442fdc1b960d764a203ad44899925acce9f64dd6"} Jul 12 15:26:30.308942 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:30.308902 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtq8d" event={"ID":"733d8099-0f3f-4195-a5fd-4813d320ce42","Type":"ContainerStarted","Data":"97b8738544c41fd0387e7dd913b03f4020bd115082add4a304a841f7e5add11e"} Jul 12 15:26:30.325366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:30.325317 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dtq8d" podStartSLOduration=375.107068472 podStartE2EDuration="6m16.325300252s" totalImagesPullingTime="1.21823178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:26:30.32416468 +0000 UTC m=+375.874365470" watchObservedRunningTime="2026-07-12 15:26:30.325300252 +0000 UTC m=+375.875501026" Jul 12 15:26:30.409836 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:30.409809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:30.465736 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:30.465703 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.262739 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.262700 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.356604 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.356565 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.428799 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.428764 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.530285 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.530208 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.629967 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.629936 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:26:38.684101 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:26:38.684069 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:02.597584 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:02.597545 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:02.650561 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:02.650529 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:34.174912 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:34.174883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:34.226900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:34.226871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:42.479907 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:42.479827 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:42.531295 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:42.531265 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:46.145077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:46.145040 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:46.200791 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:46.200762 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:53.578554 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:53.578520 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:53.626133 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:53.626097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:53.679251 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:53.679214 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:53.728939 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:53.728911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:59.027315 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:59.027283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:59.122255 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:59.122211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:27:59.448454 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:27:59.448421 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:05.587210 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:05.587174 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:19.880143 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:19.880105 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:19.979674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:19.979639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:20.468249 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.468222 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn"] Jul 12 15:28:20.468480 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.468467 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" containerName="registry" Jul 12 15:28:20.468531 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.468481 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" containerName="registry" Jul 12 15:28:20.468531 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.468521 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="723e2194-9110-4c01-a873-c02184ea3cc0" containerName="registry" Jul 12 15:28:20.471227 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.471212 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.473775 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.473749 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 15:28:20.473775 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.473771 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xz5m9\"" Jul 12 15:28:20.474716 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.474699 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 15:28:20.479445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.479424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn"] Jul 12 15:28:20.605348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.605311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.605348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.605354 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.605597 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.605375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffqw\" (UniqueName: \"kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.706697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.706655 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffqw\" (UniqueName: \"kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.706919 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.706728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.706919 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.706749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.707047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.707033 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.707126 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.707110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.715510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.715481 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffqw\" (UniqueName: \"kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.780564 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.780475 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:20.910353 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:20.910321 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn"] Jul 12 15:28:20.913430 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:28:20.913386 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acb4c60_38b2_4dec_8b20_3fb7b45e44ff.slice/crio-973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88 WatchSource:0}: Error finding container 973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88: Status 404 returned error can't find the container with id 973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88 Jul 12 15:28:21.679876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:21.679842 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" event={"ID":"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff","Type":"ContainerStarted","Data":"973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88"} Jul 12 15:28:27.645639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:27.645556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:27.703047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:27.703014 2563 generic.go:356] "Generic (PLEG): container finished" podID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerID="998bc9b97c42ce3a959bc958f836b28476d89f64ec395b430c7e162575d9d1fd" exitCode=0 Jul 12 15:28:27.703213 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:27.703108 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" event={"ID":"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff","Type":"ContainerDied","Data":"998bc9b97c42ce3a959bc958f836b28476d89f64ec395b430c7e162575d9d1fd"} Jul 12 15:28:30.714876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:30.714841 2563 generic.go:356] "Generic (PLEG): container finished" podID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerID="d92b6a239688c1895dfd9590db586b4a22e90b008bf3b4478b089a4d9686c4fe" exitCode=0 Jul 12 15:28:30.714876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:30.714881 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" event={"ID":"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff","Type":"ContainerDied","Data":"d92b6a239688c1895dfd9590db586b4a22e90b008bf3b4478b089a4d9686c4fe"} Jul 12 15:28:37.748155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:37.748062 2563 generic.go:356] "Generic (PLEG): container finished" podID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerID="f5769d905cd8356b6ac52e0d3e8bd56be904575df58397fd49005f508d89d292" exitCode=0 Jul 12 15:28:37.748155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:37.748136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" event={"ID":"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff","Type":"ContainerDied","Data":"f5769d905cd8356b6ac52e0d3e8bd56be904575df58397fd49005f508d89d292"} Jul 12 15:28:38.879884 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:38.879857 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:39.051957 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.051867 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle\") pod \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " Jul 12 15:28:39.051957 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.051915 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util\") pod \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " Jul 12 15:28:39.051957 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.051956 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sffqw\" (UniqueName: \"kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw\") pod \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\" (UID: \"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff\") " Jul 12 15:28:39.052543 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.052517 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle" (OuterVolumeSpecName: "bundle") pod "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" (UID: "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:28:39.054247 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.054214 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw" (OuterVolumeSpecName: "kube-api-access-sffqw") pod "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" (UID: "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff"). InnerVolumeSpecName "kube-api-access-sffqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:28:39.056209 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.056188 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util" (OuterVolumeSpecName: "util") pod "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" (UID: "0acb4c60-38b2-4dec-8b20-3fb7b45e44ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:28:39.153053 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.153000 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-util\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:28:39.153053 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.153048 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sffqw\" (UniqueName: \"kubernetes.io/projected/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-kube-api-access-sffqw\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:28:39.153053 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.153063 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0acb4c60-38b2-4dec-8b20-3fb7b45e44ff-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:28:39.757201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.757167 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" event={"ID":"0acb4c60-38b2-4dec-8b20-3fb7b45e44ff","Type":"ContainerDied","Data":"973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88"} Jul 12 15:28:39.757382 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.757219 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404xkhn" Jul 12 15:28:39.757382 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:39.757221 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="973bcd36ea649efca367b310b153c2854c6d822ff5875014bef668478b81fb88" Jul 12 15:28:41.443547 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:41.443509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:42.286966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:42.286936 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:45.071103 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:45.071060 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:45.134321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:45.134290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:47.778659 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:47.778626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:47.876367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:47.876337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:48.046612 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:48.046533 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:48.161570 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:48.161535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:49.806489 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:49.806443 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:50.169709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:50.169658 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:51.640518 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:51.640483 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:51.774987 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:51.774950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:56.985107 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:56.985076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:57.038459 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:57.038427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:58.580060 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:58.580027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:28:59.433453 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433423 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cpr99"] Jul 12 15:28:59.433650 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433639 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="util" Jul 12 15:28:59.433697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433652 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="util" Jul 12 15:28:59.433697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433659 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="pull" Jul 12 15:28:59.433697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433664 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="pull" Jul 12 15:28:59.433697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433671 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="extract" Jul 12 15:28:59.433697 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433677 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="extract" Jul 12 15:28:59.433838 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.433715 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0acb4c60-38b2-4dec-8b20-3fb7b45e44ff" containerName="extract" Jul 12 15:28:59.436643 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.436628 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.439216 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.439186 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 15:28:59.440258 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.440238 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-964hf\"" Jul 12 15:28:59.440352 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.440239 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 15:28:59.445593 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.445570 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cpr99"] Jul 12 15:28:59.498161 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.498131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-bound-sa-token\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.498161 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.498164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4h5x\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-kube-api-access-q4h5x\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.598963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.598925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-bound-sa-token\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.598963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.598967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4h5x\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-kube-api-access-q4h5x\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.607252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.607219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-bound-sa-token\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.607446 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.607425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4h5x\" (UniqueName: \"kubernetes.io/projected/887308c7-4172-4629-8dac-8ca812200d28-kube-api-access-q4h5x\") pod \"cert-manager-55b4d954d5-cpr99\" (UID: \"887308c7-4172-4629-8dac-8ca812200d28\") " pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.746191 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.746096 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cpr99" Jul 12 15:28:59.868292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:28:59.868267 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cpr99"] Jul 12 15:28:59.870816 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:28:59.870794 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod887308c7_4172_4629_8dac_8ca812200d28.slice/crio-33c2aba69fc785d30e15055aec925b951d1c003e126009e151e32ac2399f0b9b WatchSource:0}: Error finding container 33c2aba69fc785d30e15055aec925b951d1c003e126009e151e32ac2399f0b9b: Status 404 returned error can't find the container with id 33c2aba69fc785d30e15055aec925b951d1c003e126009e151e32ac2399f0b9b Jul 12 15:29:00.834366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:00.834331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cpr99" event={"ID":"887308c7-4172-4629-8dac-8ca812200d28","Type":"ContainerStarted","Data":"33c2aba69fc785d30e15055aec925b951d1c003e126009e151e32ac2399f0b9b"} Jul 12 15:29:02.843200 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:02.843111 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cpr99" event={"ID":"887308c7-4172-4629-8dac-8ca812200d28","Type":"ContainerStarted","Data":"4089886384a23d0806f39471d9cbf45c01b2ed2ff976b0d148e4f1f9c99abebd"} Jul 12 15:29:02.865272 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:02.865225 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cpr99" podStartSLOduration=1.237098019 podStartE2EDuration="3.865201761s" totalImagesPullingTime="2.628103742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:29:02.863871918 +0000 UTC m=+528.414072703" watchObservedRunningTime="2026-07-12 15:29:02.865201761 +0000 UTC m=+528.415402536" Jul 12 15:29:09.159573 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:09.159497 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:09.257140 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:09.257108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:14.520884 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:14.520854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:14.620222 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:14.620192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:16.387915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.387878 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv"] Jul 12 15:29:16.389876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.389860 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.392286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.392265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xz5m9\"" Jul 12 15:29:16.392425 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.392282 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 15:29:16.393364 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.393349 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 15:29:16.398501 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.398474 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv"] Jul 12 15:29:16.411364 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.411343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.411507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.411440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.411507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.411476 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7qpr\" (UniqueName: \"kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.512650 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.512609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.512845 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.512664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.512845 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.512692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7qpr\" (UniqueName: \"kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.513018 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.512997 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.513083 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.513043 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.520965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.520934 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7qpr\" (UniqueName: \"kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.699459 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.699422 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:16.820641 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.820613 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv"] Jul 12 15:29:16.824508 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:29:16.824479 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9641776_b7a6_46a3_b2ac_6e8d38e3ac91.slice/crio-243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389 WatchSource:0}: Error finding container 243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389: Status 404 returned error can't find the container with id 243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389 Jul 12 15:29:16.896805 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.896758 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerStarted","Data":"41228fd28561e6ef935763b210c88d103441d53da3c3d32a505befd59c8adc65"} Jul 12 15:29:16.896965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.896845 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerStarted","Data":"243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389"} Jul 12 15:29:16.925421 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:16.925380 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:17.902192 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:17.902108 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerID="41228fd28561e6ef935763b210c88d103441d53da3c3d32a505befd59c8adc65" exitCode=0 Jul 12 15:29:17.902192 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:17.902180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerDied","Data":"41228fd28561e6ef935763b210c88d103441d53da3c3d32a505befd59c8adc65"} Jul 12 15:29:18.908207 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:18.908172 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerID="e1ed2b6095b6b76d793206cc0f5275e7708296c34f89c7675c18975c58e10bce" exitCode=0 Jul 12 15:29:18.908674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:18.908222 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerDied","Data":"e1ed2b6095b6b76d793206cc0f5275e7708296c34f89c7675c18975c58e10bce"} Jul 12 15:29:19.914035 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:19.913998 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerID="54a1737f9b2ad173603e6ea0ead5a045fdc5ef718ceb0afe8aa0d8b29b437ea1" exitCode=0 Jul 12 15:29:19.914422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:19.914075 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerDied","Data":"54a1737f9b2ad173603e6ea0ead5a045fdc5ef718ceb0afe8aa0d8b29b437ea1"} Jul 12 15:29:21.048588 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.048563 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:21.119452 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.119414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:21.146232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.146200 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7qpr\" (UniqueName: \"kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr\") pod \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " Jul 12 15:29:21.146430 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.146253 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util\") pod \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " Jul 12 15:29:21.146430 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.146283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle\") pod \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\" (UID: \"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91\") " Jul 12 15:29:21.147022 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.146987 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle" (OuterVolumeSpecName: "bundle") pod "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" (UID: "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:29:21.148601 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.148572 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr" (OuterVolumeSpecName: "kube-api-access-b7qpr") pod "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" (UID: "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91"). InnerVolumeSpecName "kube-api-access-b7qpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:29:21.152264 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.152244 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util" (OuterVolumeSpecName: "util") pod "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" (UID: "c9641776-b7a6-46a3-b2ac-6e8d38e3ac91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:29:21.211786 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.211699 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc"] Jul 12 15:29:21.212026 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212011 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="util" Jul 12 15:29:21.212065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212029 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="util" Jul 12 15:29:21.212065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212039 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="pull" Jul 12 15:29:21.212065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212044 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="pull" Jul 12 15:29:21.212065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212058 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="extract" Jul 12 15:29:21.212065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212063 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="extract" Jul 12 15:29:21.212250 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212112 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9641776-b7a6-46a3-b2ac-6e8d38e3ac91" containerName="extract" Jul 12 15:29:21.212367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.212344 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:21.215212 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.215197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.217938 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.217906 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 15:29:21.217938 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.217925 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:29:21.218127 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.217946 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 15:29:21.218127 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.217974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 15:29:21.218127 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.217998 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 15:29:21.218283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.218182 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vnxtz\"" Jul 12 15:29:21.223676 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.223651 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc"] Jul 12 15:29:21.247238 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwgxn\" (UniqueName: \"kubernetes.io/projected/09e59495-b659-4129-a4cf-449c3a3c50dc-kube-api-access-fwgxn\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.247370 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.247370 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.247370 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247333 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/09e59495-b659-4129-a4cf-449c3a3c50dc-manager-config\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.247510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247384 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7qpr\" (UniqueName: \"kubernetes.io/projected/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-kube-api-access-b7qpr\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:29:21.247510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247415 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-util\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:29:21.247510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.247424 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9641776-b7a6-46a3-b2ac-6e8d38e3ac91-bundle\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:29:21.348317 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.348278 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.348317 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.348320 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.348562 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.348338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/09e59495-b659-4129-a4cf-449c3a3c50dc-manager-config\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.348562 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.348370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwgxn\" (UniqueName: \"kubernetes.io/projected/09e59495-b659-4129-a4cf-449c3a3c50dc-kube-api-access-fwgxn\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.349082 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.349061 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/09e59495-b659-4129-a4cf-449c3a3c50dc-manager-config\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.350916 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.350894 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.351009 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.350960 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/09e59495-b659-4129-a4cf-449c3a3c50dc-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.356714 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.356689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwgxn\" (UniqueName: \"kubernetes.io/projected/09e59495-b659-4129-a4cf-449c3a3c50dc-kube-api-access-fwgxn\") pod \"lws-controller-manager-59c7cf6567-xtqvc\" (UID: \"09e59495-b659-4129-a4cf-449c3a3c50dc\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.524812 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.524720 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:21.665721 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.665535 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc"] Jul 12 15:29:21.668245 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:29:21.668219 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e59495_b659_4129_a4cf_449c3a3c50dc.slice/crio-ec05b810c18a6a0bbedf6f703ce95a287a1fd95610bb22bf963b5f59d262a459 WatchSource:0}: Error finding container ec05b810c18a6a0bbedf6f703ce95a287a1fd95610bb22bf963b5f59d262a459: Status 404 returned error can't find the container with id ec05b810c18a6a0bbedf6f703ce95a287a1fd95610bb22bf963b5f59d262a459 Jul 12 15:29:21.923175 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.923146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" event={"ID":"c9641776-b7a6-46a3-b2ac-6e8d38e3ac91","Type":"ContainerDied","Data":"243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389"} Jul 12 15:29:21.923349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.923198 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="243ae873655a8b610b09c94a7e144cba7802766b11efaf5dd6b235e7316f5389" Jul 12 15:29:21.923349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.923158 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afglsgv" Jul 12 15:29:21.924272 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:21.924251 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" event={"ID":"09e59495-b659-4129-a4cf-449c3a3c50dc","Type":"ContainerStarted","Data":"ec05b810c18a6a0bbedf6f703ce95a287a1fd95610bb22bf963b5f59d262a459"} Jul 12 15:29:24.942678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:24.942644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" event={"ID":"09e59495-b659-4129-a4cf-449c3a3c50dc","Type":"ContainerStarted","Data":"3ef445e0ded86611ecebb81e7b4b0e294f6636ba9f348f261b3a4986aa80cffe"} Jul 12 15:29:24.943077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:24.942767 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:24.959864 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:24.959814 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" podStartSLOduration=1.412187327 podStartE2EDuration="3.959799788s" totalImagesPullingTime="2.547612461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:29:24.957970237 +0000 UTC m=+550.508171011" watchObservedRunningTime="2026-07-12 15:29:24.959799788 +0000 UTC m=+550.510000562" Jul 12 15:29:32.318742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:32.318714 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.344542 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.344515 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.445406 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.445374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.508016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.507988 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v"] Jul 12 15:29:33.511218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.511203 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.516640 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.516612 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 15:29:33.516804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.516665 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 15:29:33.516804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.516778 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4sjfz\"" Jul 12 15:29:33.517011 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.516995 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 15:29:33.517108 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.517091 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 15:29:33.536490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.536451 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v"] Jul 12 15:29:33.537438 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.537388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwn9c\" (UniqueName: \"kubernetes.io/projected/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-kube-api-access-xwn9c\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.537537 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.537481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.537537 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.537520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.537624 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.537546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.537624 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.537592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.622095 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.622012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.638111 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.638244 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.638244 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638145 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.638244 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.638424 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638243 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwn9c\" (UniqueName: \"kubernetes.io/projected/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-kube-api-access-xwn9c\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.638539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.638514 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.640716 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.640689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.640816 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.640699 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.640861 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.640837 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.646202 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.646177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwn9c\" (UniqueName: \"kubernetes.io/projected/422bc9f3-bcd3-4270-bb2d-41b96b2e6beb-kube-api-access-xwn9c\") pod \"opendatahub-operator-controller-manager-7558dcb54-lw62v\" (UID: \"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.723436 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.723381 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.821349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.821308 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:33.828445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.828417 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.922674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.922637 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:33.960535 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.960513 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v"] Jul 12 15:29:33.963315 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:29:33.963291 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422bc9f3_bcd3_4270_bb2d_41b96b2e6beb.slice/crio-f2c82e123e87900b6f1859072b8bce76fb26974b99982a82878e3154a74dcb55 WatchSource:0}: Error finding container f2c82e123e87900b6f1859072b8bce76fb26974b99982a82878e3154a74dcb55: Status 404 returned error can't find the container with id f2c82e123e87900b6f1859072b8bce76fb26974b99982a82878e3154a74dcb55 Jul 12 15:29:33.973525 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:33.973501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" event={"ID":"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb","Type":"ContainerStarted","Data":"f2c82e123e87900b6f1859072b8bce76fb26974b99982a82878e3154a74dcb55"} Jul 12 15:29:34.032462 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:34.032426 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:34.127829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:34.127790 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:35.948579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:35.948538 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-xtqvc" Jul 12 15:29:36.988229 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:36.988201 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" event={"ID":"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb","Type":"ContainerStarted","Data":"c9942822e34668ab6b92f1e3a61580fe9d16cd0f5d010cdd32d970efadf35e1f"} Jul 12 15:29:37.993893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:37.993860 2563 generic.go:356] "Generic (PLEG): container finished" podID="422bc9f3-bcd3-4270-bb2d-41b96b2e6beb" containerID="c9942822e34668ab6b92f1e3a61580fe9d16cd0f5d010cdd32d970efadf35e1f" exitCode=0 Jul 12 15:29:37.994338 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:37.993899 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" event={"ID":"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb","Type":"ContainerDied","Data":"c9942822e34668ab6b92f1e3a61580fe9d16cd0f5d010cdd32d970efadf35e1f"} Jul 12 15:29:38.999043 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:38.999008 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" event={"ID":"422bc9f3-bcd3-4270-bb2d-41b96b2e6beb","Type":"ContainerStarted","Data":"383550d3ccf4498ea4200c7fcdb5b4b3789ab824d278aec13b565fc8aacac1a1"} Jul 12 15:29:38.999440 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:38.999153 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:39.021248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:39.021175 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" podStartSLOduration=1.797835129 podStartE2EDuration="6.021152679s" totalImagesPullingTime="3.167735781s" totalInitContainerRuntime="1.055581769s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:29:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:29:39.019265261 +0000 UTC m=+564.569466035" watchObservedRunningTime="2026-07-12 15:29:39.021152679 +0000 UTC m=+564.571353456" Jul 12 15:29:45.324174 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:45.324132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:45.424178 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:45.424145 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:50.005906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:50.005870 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-lw62v" Jul 12 15:29:52.585870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:52.585801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:52.707453 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:52.707414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:53.411283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:53.411246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:53.479898 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:53.479853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:53.580715 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:53.580647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:53.685495 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:53.685367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:53.914474 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:53.914422 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:54.005683 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:54.005572 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:55.687320 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:55.687259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:56.053579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:56.053482 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:56.869488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:56.869451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:56.930804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:56.930773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:29:57.720445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:29:57.720390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:00.291825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.291788 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k"] Jul 12 15:30:00.303908 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.303878 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.306873 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.306847 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k"] Jul 12 15:30:00.307146 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307124 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 15:30:00.307276 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 15:30:00.307276 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307252 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 15:30:00.307276 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307144 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 15:30:00.307480 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307140 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9br64\"" Jul 12 15:30:00.307480 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307185 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 15:30:00.307480 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.307215 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 15:30:00.431123 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431123 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431165 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431208 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jltwk\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-kube-api-access-jltwk\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.431379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.431298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltwk\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-kube-api-access-jltwk\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.531988 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.531879 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.532684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.532546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.534275 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.534251 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.534365 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.534304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.534466 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.534446 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.534507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.534471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.540668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.540644 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.541091 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.541075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltwk\" (UniqueName: \"kubernetes.io/projected/095647c1-a4c1-4b34-88f5-e58a51e4bb2f-kube-api-access-jltwk\") pod \"istiod-openshift-gateway-94bb8fbfd-4wq5k\" (UID: \"095647c1-a4c1-4b34-88f5-e58a51e4bb2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.614947 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.614852 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:00.754087 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:00.754063 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k"] Jul 12 15:30:00.755838 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:30:00.755807 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095647c1_a4c1_4b34_88f5_e58a51e4bb2f.slice/crio-4f58e7c04b5b8b1739cd642e91e9932d7f95914433aa48e8d03b47278e903700 WatchSource:0}: Error finding container 4f58e7c04b5b8b1739cd642e91e9932d7f95914433aa48e8d03b47278e903700: Status 404 returned error can't find the container with id 4f58e7c04b5b8b1739cd642e91e9932d7f95914433aa48e8d03b47278e903700 Jul 12 15:30:01.083224 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:01.083190 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" event={"ID":"095647c1-a4c1-4b34-88f5-e58a51e4bb2f","Type":"ContainerStarted","Data":"4f58e7c04b5b8b1739cd642e91e9932d7f95914433aa48e8d03b47278e903700"} Jul 12 15:30:01.400391 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:01.400300 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:01.539319 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:01.539286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:02.551157 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:02.551122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:02.751723 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:02.751690 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:02.912275 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:02.912243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:03.176260 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:03.176169 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:03.273492 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:03.273451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:03.477322 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:03.477222 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:04.420490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:04.420457 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:04.579467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:04.579432 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:07.203693 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:07.203657 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:30:07.203945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:07.203734 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:30:08.111788 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:08.111753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" event={"ID":"095647c1-a4c1-4b34-88f5-e58a51e4bb2f","Type":"ContainerStarted","Data":"5cab9bdc37ad8a8eec161d77e4299b8770c12df19cbaa73c23797d4f067029f2"} Jul 12 15:30:08.111979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:08.111958 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:08.113526 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:08.113508 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" Jul 12 15:30:08.148897 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:08.148843 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4wq5k" podStartSLOduration=1.703719722 podStartE2EDuration="8.148827309s" totalImagesPullingTime="6.445107587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:30:08.148245395 +0000 UTC m=+593.698446189" watchObservedRunningTime="2026-07-12 15:30:08.148827309 +0000 UTC m=+593.699028083" Jul 12 15:30:14.821876 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:14.821848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:30:14.822378 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:14.822356 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:30:36.046564 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:36.046488 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:36.101690 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:36.101660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:54.042617 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:54.042583 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:54.136706 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:54.136677 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:54.209349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:54.209316 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:54.965942 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:54.965898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:30:55.173428 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:30:55.173336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:14.992173 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:14.992129 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:23.407102 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:23.407070 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:27.612233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:27.612144 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:28.008223 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:28.008187 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:28.099915 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:28.099882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:30.011645 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:30.011591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:30.606730 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:30.606700 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:30.860490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:30.860379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:32.620460 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:32.620393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:32.722210 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:32.722177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:33.820727 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:33.820694 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:33.932960 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:33.932921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:35.898346 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:35.898312 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:38.763174 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:38.763137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:38.863211 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:38.863183 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:38.961122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:38.961094 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:39.067969 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:39.067866 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:39.162070 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:39.162039 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:42.413207 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:42.413165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:42.514887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:42.514850 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:46.989606 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:46.989561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:47.091659 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:47.091626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:48.216368 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:48.216342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:48.319424 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:48.319370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:48.420935 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:48.420893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:48.538923 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:48.538828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:51.554058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:51.554012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:51.655618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:51.655578 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:58.205672 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:58.205631 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:31:58.305411 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:31:58.305364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:01.192951 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:01.192913 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:01.294445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:01.294413 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:01.557664 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:01.557567 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:01.657153 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:01.657119 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:06.895910 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:06.895797 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:06.997589 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:06.997550 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:07.126305 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:07.126274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:07.593473 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:07.593437 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:07.651164 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:07.651136 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:09.864742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:09.864699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:09.952973 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:09.952934 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:26.074667 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:26.074635 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:26.499634 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:26.499602 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:39.937985 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:39.937944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:40.041917 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.041867 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:40.138028 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.137996 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:40.302514 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.302437 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:32:40.305654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.305634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.308163 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.308139 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 15:32:40.308280 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.308251 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 15:32:40.308331 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.308286 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 15:32:40.309268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.309252 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2bhgs\"" Jul 12 15:32:40.314544 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.314526 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:32:40.325216 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.325198 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-config-file\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.325314 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.325236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzvlg\" (UniqueName: \"kubernetes.io/projected/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-kube-api-access-tzvlg\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.338674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.338654 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:32:40.426250 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.426215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-config-file\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.426485 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.426263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzvlg\" (UniqueName: \"kubernetes.io/projected/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-kube-api-access-tzvlg\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.426962 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.426935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-config-file\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.436992 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.436958 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzvlg\" (UniqueName: \"kubernetes.io/projected/c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6-kube-api-access-tzvlg\") pod \"limitador-limitador-54d6d988d5-wg68v\" (UID: \"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.616142 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.616041 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:40.741391 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.737806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:40.749379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.749341 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:32:40.753198 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:32:40.753160 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc166bdb8_fe5d_44e0_8c89_74a6dba3a3e6.slice/crio-75c6d65100160bb5901361a2ef5e55abcd5d0c88bd85b2d4bd0a410e7670d3e5 WatchSource:0}: Error finding container 75c6d65100160bb5901361a2ef5e55abcd5d0c88bd85b2d4bd0a410e7670d3e5: Status 404 returned error can't find the container with id 75c6d65100160bb5901361a2ef5e55abcd5d0c88bd85b2d4bd0a410e7670d3e5 Jul 12 15:32:40.754903 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.754888 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:32:40.804442 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:40.804377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:32:41.735292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:41.735254 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" event={"ID":"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6","Type":"ContainerStarted","Data":"75c6d65100160bb5901361a2ef5e55abcd5d0c88bd85b2d4bd0a410e7670d3e5"} Jul 12 15:32:43.745349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:43.745310 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" event={"ID":"c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6","Type":"ContainerStarted","Data":"ed0739f9ddb06698391df463df89ad25c3c886667f26e3a3c1d5e147a687ab13"} Jul 12 15:32:43.745755 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:43.745471 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:43.761520 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:43.761469 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" podStartSLOduration=1.253826968 podStartE2EDuration="3.761452253s" totalImagesPullingTime="2.507625285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:32:43.760025834 +0000 UTC m=+749.310226629" watchObservedRunningTime="2026-07-12 15:32:43.761452253 +0000 UTC m=+749.311653027" Jul 12 15:32:54.753197 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:54.753159 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wg68v" Jul 12 15:32:59.680526 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:32:59.680494 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:12.420547 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:12.420513 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:12.649317 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:12.649281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:18.277069 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:18.277035 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:18.377281 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:18.377244 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:18.439899 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:18.439871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:18.530035 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:18.529949 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:21.032976 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:21.032940 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:21.126655 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:21.126626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:44.852871 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:44.852793 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:44.923681 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:44.923641 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:50.233262 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:50.233216 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:50.646258 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:50.646185 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:50.869018 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:50.868986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:51.128004 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:51.127977 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:51.269610 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:51.269580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:51.454696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:51.454662 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:33:57.787730 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:57.787695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:33:57.888810 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:33:57.888775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:00.483743 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:00.483704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:00.555825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:00.555786 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:08.928649 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:08.928608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:09.388437 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:09.388337 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:34:25.722159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:25.722121 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:26.554321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:26.554294 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:34:32.324734 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:32.324697 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:33.393031 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:33.392994 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:34:44.698902 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.698872 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz"] Jul 12 15:34:44.701753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.701733 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.704481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.704458 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 15:34:44.705723 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.705700 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 12 15:34:44.705883 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.705723 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nqvwk\"" Jul 12 15:34:44.705883 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.705727 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 15:34:44.715095 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.715069 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz"] Jul 12 15:34:44.738763 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.738731 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:44.757626 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.757735 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.757735 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.757802 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.757834 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757801 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgch9\" (UniqueName: \"kubernetes.io/projected/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-kube-api-access-fgch9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.757834 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.757828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.858819 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.858819 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgch9\" (UniqueName: \"kubernetes.io/projected/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-kube-api-access-fgch9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.858966 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859246 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.859223 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859353 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.859334 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.859437 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.859361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.861330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.861300 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.861585 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.861569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:44.867903 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:44.867884 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgch9\" (UniqueName: \"kubernetes.io/projected/0ef87217-8dea-46b9-b2f6-b9d3d290ea00-kube-api-access-fgch9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz\" (UID: \"0ef87217-8dea-46b9-b2f6-b9d3d290ea00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:45.012039 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:45.011955 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:45.143674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:45.143650 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz"] Jul 12 15:34:45.146209 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:34:45.146180 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef87217_8dea_46b9_b2f6_b9d3d290ea00.slice/crio-fec99d32e080a710badd379c03a0f2574b1db9c0c3171b8df7fbc728c3878131 WatchSource:0}: Error finding container fec99d32e080a710badd379c03a0f2574b1db9c0c3171b8df7fbc728c3878131: Status 404 returned error can't find the container with id fec99d32e080a710badd379c03a0f2574b1db9c0c3171b8df7fbc728c3878131 Jul 12 15:34:45.245258 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:45.245219 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" event={"ID":"0ef87217-8dea-46b9-b2f6-b9d3d290ea00","Type":"ContainerStarted","Data":"fec99d32e080a710badd379c03a0f2574b1db9c0c3171b8df7fbc728c3878131"} Jul 12 15:34:46.142010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:46.141984 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:34:46.251126 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:46.251095 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" event={"ID":"0ef87217-8dea-46b9-b2f6-b9d3d290ea00","Type":"ContainerStarted","Data":"5236a12ec7f9e67310bfb2c313f99468ceab4e770e0e3abdca8d1145882c0807"} Jul 12 15:34:46.271453 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:46.271378 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" podStartSLOduration=1.296093057 podStartE2EDuration="2.271358984s" totalImagesPullingTime="975.265927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:34:46.270317551 +0000 UTC m=+871.820518344" watchObservedRunningTime="2026-07-12 15:34:46.271358984 +0000 UTC m=+871.821559760" Jul 12 15:34:50.200372 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:50.200335 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:34:53.156058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:53.156028 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:34:55.012414 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:55.012379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:55.012799 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:55.012445 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:55.025068 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:55.025044 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:55.298198 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:55.298129 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz" Jul 12 15:34:58.050185 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:34:58.050132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:35:00.449825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:00.449791 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:35:14.844379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:14.844352 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:35:14.845854 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:14.845834 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:35:51.367320 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:51.367284 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:35:51.456020 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:51.455991 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:35:52.457428 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:52.457378 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:35:52.555751 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:35:52.555721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:36:22.359709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:36:22.359674 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:36:32.545711 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:36:32.545671 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:36:43.048760 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:36:43.048678 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:36:53.249729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:36:53.249686 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:37:03.649092 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:03.649055 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:37:05.863976 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:05.863940 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:07.315726 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:07.315692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:31.144058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:31.144023 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:37:41.948032 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:41.947994 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:37:43.740080 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:43.740038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:47.719893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:47.719854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:47.816327 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:47.816297 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:47.916367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:47.916335 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:53.602935 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:53.602897 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:53.705113 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:53.705084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:53.804151 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:53.804113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:53.944478 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:53.944447 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:59.482361 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:59.482327 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:37:59.583220 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:37:59.583189 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:38:15.339707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:15.339670 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:38:24.238635 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:24.238598 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:38:24.595642 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:24.595555 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:38:32.939601 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:32.939563 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:38:39.238089 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:39.238054 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:38:50.036419 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:50.036363 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:38:59.038598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:38:59.038566 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:10.440070 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:10.440033 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:16.538421 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:16.538369 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:19.406746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:19.406712 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:39:24.137629 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:24.137596 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:33.741678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:33.741637 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:45.741921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:45.741836 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:39:54.734948 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:39:54.734912 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:40:14.868282 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:14.868255 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:40:14.871182 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:14.871161 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:40:17.141100 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:17.141062 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:40:26.843257 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:26.843219 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:40:45.148007 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:45.147973 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:40:53.940200 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:40:53.940162 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:41:02.947466 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:41:02.947427 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:41:12.138268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:41:12.138189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:41:20.345686 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:41:20.345642 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:42:38.920345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:42:38.920264 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:43:13.033271 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:43:13.033234 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:43:21.634421 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:43:21.634371 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:43:33.937699 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:43:33.937667 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:43:42.835910 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:43:42.835868 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:43:51.845627 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:43:51.845590 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:44:01.048065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:44:01.048028 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:44:14.337012 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:44:14.336976 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:44:22.943045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:44:22.943013 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:44:32.140794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:44:32.140755 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:44:42.633517 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:44:42.633479 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:45:00.140430 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.140378 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29731185-l5ssh"] Jul 12 15:45:00.143739 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.143717 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:00.146042 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.146024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:45:00.156375 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.156348 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29731185-l5ssh"] Jul 12 15:45:00.229555 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.229519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r6mt\" (UniqueName: \"kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt\") pod \"maas-api-key-cleanup-29731185-l5ssh\" (UID: \"a5012b28-5c9d-4949-a0c1-6b77763e84e2\") " pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:00.330075 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.330037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r6mt\" (UniqueName: \"kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt\") pod \"maas-api-key-cleanup-29731185-l5ssh\" (UID: \"a5012b28-5c9d-4949-a0c1-6b77763e84e2\") " pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:00.338195 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.338170 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r6mt\" (UniqueName: \"kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt\") pod \"maas-api-key-cleanup-29731185-l5ssh\" (UID: \"a5012b28-5c9d-4949-a0c1-6b77763e84e2\") " pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:00.453486 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.453439 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:00.584146 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.584047 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29731185-l5ssh"] Jul 12 15:45:00.587132 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:45:00.587104 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5012b28_5c9d_4949_a0c1_6b77763e84e2.slice/crio-e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040 WatchSource:0}: Error finding container e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040: Status 404 returned error can't find the container with id e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040 Jul 12 15:45:00.589445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.589424 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:45:00.756523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:00.756432 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" event={"ID":"a5012b28-5c9d-4949-a0c1-6b77763e84e2","Type":"ContainerStarted","Data":"e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040"} Jul 12 15:45:03.773261 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:03.773218 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" event={"ID":"a5012b28-5c9d-4949-a0c1-6b77763e84e2","Type":"ContainerStarted","Data":"c7069acc1780bf4c7e1339dadef57d2813ca465bfe3314f7e938c74e9413b4cf"} Jul 12 15:45:03.787608 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:03.787567 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" podStartSLOduration=1.088168035 podStartE2EDuration="3.787550818s" totalImagesPullingTime="2.699382783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:45:03.786809374 +0000 UTC m=+1489.337010159" watchObservedRunningTime="2026-07-12 15:45:03.787550818 +0000 UTC m=+1489.337751635" Jul 12 15:45:04.779716 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:04.779684 2563 generic.go:356] "Generic (PLEG): container finished" podID="a5012b28-5c9d-4949-a0c1-6b77763e84e2" containerID="c7069acc1780bf4c7e1339dadef57d2813ca465bfe3314f7e938c74e9413b4cf" exitCode=0 Jul 12 15:45:04.780086 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:04.779742 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" event={"ID":"a5012b28-5c9d-4949-a0c1-6b77763e84e2","Type":"ContainerDied","Data":"c7069acc1780bf4c7e1339dadef57d2813ca465bfe3314f7e938c74e9413b4cf"} Jul 12 15:45:05.909543 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:05.909512 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:05.977780 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:05.977747 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r6mt\" (UniqueName: \"kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt\") pod \"a5012b28-5c9d-4949-a0c1-6b77763e84e2\" (UID: \"a5012b28-5c9d-4949-a0c1-6b77763e84e2\") " Jul 12 15:45:05.983720 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:05.983577 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt" (OuterVolumeSpecName: "kube-api-access-7r6mt") pod "a5012b28-5c9d-4949-a0c1-6b77763e84e2" (UID: "a5012b28-5c9d-4949-a0c1-6b77763e84e2"). InnerVolumeSpecName "kube-api-access-7r6mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:45:06.079189 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:06.079090 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r6mt\" (UniqueName: \"kubernetes.io/projected/a5012b28-5c9d-4949-a0c1-6b77763e84e2-kube-api-access-7r6mt\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:45:06.789254 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:06.789219 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" event={"ID":"a5012b28-5c9d-4949-a0c1-6b77763e84e2","Type":"ContainerDied","Data":"e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040"} Jul 12 15:45:06.789254 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:06.789248 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29731185-l5ssh" Jul 12 15:45:06.789523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:06.789298 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e18183038e8d341ee287337a9a76df6648bf6f5f013e7a4b8b506a06a7b4b040" Jul 12 15:45:14.891666 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:14.891633 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:45:14.894492 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:14.894471 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:45:17.539822 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:17.539787 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:45:26.243913 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:26.243867 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:45:35.445314 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:35.445268 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:45:47.742345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:47.742307 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:45:54.461033 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:45:54.460998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:46:02.438902 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:02.438868 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:11.641093 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:11.641051 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:20.138866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:20.138828 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:29.842003 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:29.841967 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:38.958350 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:38.958316 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:47.838285 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:47.838241 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:46:51.941454 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:51.941415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:46:52.041150 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:46:52.041111 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:47:00.239507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:00.239471 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:47:10.040904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:10.040821 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:47:19.344684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:19.344644 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:47:31.337348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:31.337306 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:47:43.142911 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:43.142876 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:47:49.339084 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:47:49.339047 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:00.733045 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:00.733011 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:12.636801 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:12.636765 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:25.338794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:25.338762 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:28.553517 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:28.553485 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:33.645022 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:33.644990 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:37.934049 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:37.933878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:46.200767 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:46.200733 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:46.639107 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:46.639024 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:48:51.257573 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:51.257534 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:54.009372 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:54.009329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:54.126172 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:54.126134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:54.453347 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:54.453319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:48:55.048449 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:48:55.048414 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:49:00.268571 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.268541 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:00.320586 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.320553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:00.443046 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.443013 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:00.443445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.443428 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5012b28-5c9d-4949-a0c1-6b77763e84e2" containerName="cleanup" Jul 12 15:49:00.443500 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.443450 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5012b28-5c9d-4949-a0c1-6b77763e84e2" containerName="cleanup" Jul 12 15:49:00.443552 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.443541 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5012b28-5c9d-4949-a0c1-6b77763e84e2" containerName="cleanup" Jul 12 15:49:00.445678 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.445653 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.448273 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.448251 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6cf7acf5-openshift-default-dockercfg-rkw44\"" Jul 12 15:49:00.464184 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.464150 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:00.477842 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.477804 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:00.580118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.580027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:00.620027 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620083 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620115 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9xl4\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620442 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.620442 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.620294 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721249 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721213 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721249 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9xl4\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.721921 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721823 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.722066 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.721968 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.722229 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.722177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.723804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.723782 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.724122 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.724104 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.737529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.737499 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9xl4\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.737845 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.737828 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token\") pod \"e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.756719 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.756696 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:00.886871 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:00.886845 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:00.889039 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:49:00.889015 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9150403_809a_4314_849a_1ce8c92bf2c2.slice/crio-d0b4124c0b7536afeb06a6702d5b2ebfcbd23a96db4525f1cbb0b83ad6fad45e WatchSource:0}: Error finding container d0b4124c0b7536afeb06a6702d5b2ebfcbd23a96db4525f1cbb0b83ad6fad45e: Status 404 returned error can't find the container with id d0b4124c0b7536afeb06a6702d5b2ebfcbd23a96db4525f1cbb0b83ad6fad45e Jul 12 15:49:01.650411 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:01.650366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:01.724248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:01.724213 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:01.760115 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:01.760067 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" event={"ID":"d9150403-809a-4314-849a-1ce8c92bf2c2","Type":"ContainerStarted","Data":"d0b4124c0b7536afeb06a6702d5b2ebfcbd23a96db4525f1cbb0b83ad6fad45e"} Jul 12 15:49:01.792610 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:01.792580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:01.893556 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:01.893519 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:02.208450 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:02.208385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:02.312081 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:02.312045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:03.394258 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.394219 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:03.394540 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.394298 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:03.394540 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.394326 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:03.772297 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.772260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" event={"ID":"d9150403-809a-4314-849a-1ce8c92bf2c2","Type":"ContainerStarted","Data":"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17"} Jul 12 15:49:03.793349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.793300 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" podStartSLOduration=1.29039672 podStartE2EDuration="3.793284813s" totalImagesPullingTime="2.502888093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:49:03.7918212 +0000 UTC m=+1729.342021976" watchObservedRunningTime="2026-07-12 15:49:03.793284813 +0000 UTC m=+1729.343485588" Jul 12 15:49:03.948978 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:03.948952 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:04.757684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:04.757647 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:04.762877 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:04.762853 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:04.777675 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:04.777641 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:04.778310 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:04.778293 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:04.863220 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:04.863189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:49:06.765750 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:06.765721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:06.935741 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:06.935707 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:06.949709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:06.949676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:07.024677 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:07.024545 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:07.081321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:07.081291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:07.791285 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:07.791250 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" podUID="d9150403-809a-4314-849a-1ce8c92bf2c2" containerName="istio-proxy" containerID="cri-o://0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17" gracePeriod=30 Jul 12 15:49:10.157084 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:10.157053 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:12.437355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.437333 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:12.513932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.513855 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.513932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.513892 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.513932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.513918 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.513932 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.513933 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514139 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9xl4\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514193 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514195 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:12.514233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514219 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514249 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514285 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket\") pod \"d9150403-809a-4314-849a-1ce8c92bf2c2\" (UID: \"d9150403-809a-4314-849a-1ce8c92bf2c2\") " Jul 12 15:49:12.514481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514385 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:49:12.514636 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514539 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9150403-809a-4314-849a-1ce8c92bf2c2-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.514636 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514615 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:12.514636 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514623 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.514808 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514789 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:12.514865 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.514842 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:12.516602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.516574 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:49:12.516737 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.516667 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:12.516799 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.516740 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:12.516799 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.516774 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4" (OuterVolumeSpecName: "kube-api-access-h9xl4") pod "d9150403-809a-4314-849a-1ce8c92bf2c2" (UID: "d9150403-809a-4314-849a-1ce8c92bf2c2"). InnerVolumeSpecName "kube-api-access-h9xl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:12.546306 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.546277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:49:12.615283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615242 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9xl4\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-kube-api-access-h9xl4\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615272 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615288 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615567 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615299 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615567 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615311 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615567 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615323 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.615567 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.615333 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9150403-809a-4314-849a-1ce8c92bf2c2-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:12.779958 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.779869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:12.812575 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.812538 2563 generic.go:356] "Generic (PLEG): container finished" podID="d9150403-809a-4314-849a-1ce8c92bf2c2" containerID="0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17" exitCode=0 Jul 12 15:49:12.812733 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.812605 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" Jul 12 15:49:12.812733 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.812616 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" event={"ID":"d9150403-809a-4314-849a-1ce8c92bf2c2","Type":"ContainerDied","Data":"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17"} Jul 12 15:49:12.812733 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.812665 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg" event={"ID":"d9150403-809a-4314-849a-1ce8c92bf2c2","Type":"ContainerDied","Data":"d0b4124c0b7536afeb06a6702d5b2ebfcbd23a96db4525f1cbb0b83ad6fad45e"} Jul 12 15:49:12.812733 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.812690 2563 scope.go:117] "RemoveContainer" containerID="0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17" Jul 12 15:49:12.821607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.821592 2563 scope.go:117] "RemoveContainer" containerID="0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17" Jul 12 15:49:12.821894 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:49:12.821874 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17\": container with ID starting with 0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17 not found: ID does not exist" containerID="0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17" Jul 12 15:49:12.821941 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.821904 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17"} err="failed to get container status \"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17\": rpc error: code = NotFound desc = could not find container \"0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17\": container with ID starting with 0ad4359bd017905062b63de808688d7445b30ad53000173c7e302b604c43ab17 not found: ID does not exist" Jul 12 15:49:12.832934 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.832905 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:12.834784 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.834762 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6cf7acf5-openshift-default-678c799fd5-w7nvg"] Jul 12 15:49:12.838436 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.838392 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:12.929513 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.929477 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9150403-809a-4314-849a-1ce8c92bf2c2" path="/var/lib/kubelet/pods/d9150403-809a-4314-849a-1ce8c92bf2c2/volumes" Jul 12 15:49:12.939790 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.939761 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:12.940061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.940046 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9150403-809a-4314-849a-1ce8c92bf2c2" containerName="istio-proxy" Jul 12 15:49:12.940061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.940061 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9150403-809a-4314-849a-1ce8c92bf2c2" containerName="istio-proxy" Jul 12 15:49:12.940194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.940163 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9150403-809a-4314-849a-1ce8c92bf2c2" containerName="istio-proxy" Jul 12 15:49:12.944413 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.944361 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:12.946965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.946939 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5d5046a9-openshift-default-dockercfg-dlfj2\"" Jul 12 15:49:12.956911 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.956862 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:12.979918 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:12.979880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:13.019247 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019253 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019316 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svhqw\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019467 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019382 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019478 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.019729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.019554 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.079390 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.079302 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:13.120554 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120554 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120559 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svhqw\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.120966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.120967 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121117 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.121021 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.121266 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121352 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.121294 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121352 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.121338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.121473 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.121368 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.123139 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.123117 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.123516 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.123501 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.130444 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.130420 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svhqw\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.132353 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.132335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token\") pod \"e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.256918 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.256879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:13.390131 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.390103 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:13.392428 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:49:13.392387 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c33e0e8_9379_4e3a_ab50_460a09d1efa8.slice/crio-fb1496f76b2ee1b52d89715c674a0997ba2eb4d4f761868cc855589d751ae6c0 WatchSource:0}: Error finding container fb1496f76b2ee1b52d89715c674a0997ba2eb4d4f761868cc855589d751ae6c0: Status 404 returned error can't find the container with id fb1496f76b2ee1b52d89715c674a0997ba2eb4d4f761868cc855589d751ae6c0 Jul 12 15:49:13.395592 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.395558 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:13.395661 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.395627 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:13.395661 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.395654 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:13.820868 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.820824 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerStarted","Data":"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c"} Jul 12 15:49:13.821367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.820888 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerStarted","Data":"fb1496f76b2ee1b52d89715c674a0997ba2eb4d4f761868cc855589d751ae6c0"} Jul 12 15:49:13.841863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:13.841812 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" podStartSLOduration=1.841794845 podStartE2EDuration="1.841794845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:49:13.839887147 +0000 UTC m=+1739.390087937" watchObservedRunningTime="2026-07-12 15:49:13.841794845 +0000 UTC m=+1739.391995619" Jul 12 15:49:14.175049 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.175008 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.247667 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.247633 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.257194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.257151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:14.261646 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.261623 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:14.309311 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.309278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.414985 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.414951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.705913 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.705879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.805255 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.805219 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:14.826359 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.826337 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:14.827434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:14.827415 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:16.722318 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:16.722285 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:19.255220 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:19.255190 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:19.503559 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:19.503528 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:24.870354 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.870316 2563 generic.go:356] "Generic (PLEG): container finished" podID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerID="ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c" exitCode=0 Jul 12 15:49:24.870785 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.870392 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerDied","Data":"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c"} Jul 12 15:49:24.870850 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.870832 2563 scope.go:117] "RemoveContainer" containerID="ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c" Jul 12 15:49:24.871853 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.871816 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:24.871963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.871906 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:24.872038 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:24.871960 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:25.492317 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:25.492284 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:25.514305 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:25.514267 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:25.614654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:25.614622 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:25.876848 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:25.876753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerStarted","Data":"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852"} Jul 12 15:49:25.877286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:25.876893 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" containerID="cri-o://f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852" gracePeriod=30 Jul 12 15:49:27.356288 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:27.356263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:29.826617 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:29.826580 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:31.118820 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.118793 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:31.162765 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162738 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svhqw\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162770 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162786 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162805 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162848 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162871 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162897 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.162945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162929 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.163261 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.162966 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket\") pod \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\" (UID: \"5c33e0e8-9379-4e3a-ab50-460a09d1efa8\") " Jul 12 15:49:31.163261 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.163106 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:31.163364 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.163332 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:49:31.163448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.163043 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:31.163526 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.163502 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:31.163982 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.163953 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:31.165225 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.165191 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:31.165225 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.165200 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw" (OuterVolumeSpecName: "kube-api-access-svhqw") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "kube-api-access-svhqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:31.165708 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.165689 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:31.165763 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.165705 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c33e0e8-9379-4e3a-ab50-460a09d1efa8" (UID: "5c33e0e8-9379-4e3a-ab50-460a09d1efa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:49:31.210575 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.210542 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:31.264520 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264487 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264520 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264517 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264520 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264526 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svhqw\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-kube-api-access-svhqw\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264537 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264545 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264554 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264561 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264568 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.264738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.264576 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c33e0e8-9379-4e3a-ab50-460a09d1efa8-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:31.310944 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.310903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:31.377496 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.377466 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:31.377741 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.377726 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:49:31.377741 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.377740 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:49:31.377866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.377801 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:49:31.377866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.377810 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:49:31.381004 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.380981 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.383426 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.383386 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7ed6f425-openshift-default-dockercfg-xlg4v\"" Jul 12 15:49:31.395358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.395322 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:31.417336 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.417262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:31.466366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466390 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrg2\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466550 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466715 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466715 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.466715 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.466677 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.479925 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.479894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:31.567945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.567901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrg2\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.567945 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.567944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.567965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.567989 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568012 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568525 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568597 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568721 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568664 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.568721 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.568662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.569075 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.569053 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.570435 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.570389 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.570676 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.570657 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.575908 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.575886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrg2\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.576036 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.575963 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token\") pod \"e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.700368 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.700267 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:31.832607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.832578 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:31.835243 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:49:31.835219 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17b6456_069d_4581_b14d_f6f6be888bc3.slice/crio-41efd0b7038b9893060738a54470cf5873812ef376761a6013ac01779e5ab31a WatchSource:0}: Error finding container 41efd0b7038b9893060738a54470cf5873812ef376761a6013ac01779e5ab31a: Status 404 returned error can't find the container with id 41efd0b7038b9893060738a54470cf5873812ef376761a6013ac01779e5ab31a Jul 12 15:49:31.837217 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.837187 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:31.837269 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.837250 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:31.837312 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.837279 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:49:31.903258 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.903226 2563 generic.go:356] "Generic (PLEG): container finished" podID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerID="f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852" exitCode=0 Jul 12 15:49:31.903378 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.903274 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" Jul 12 15:49:31.903378 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.903306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerDied","Data":"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852"} Jul 12 15:49:31.903378 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.903371 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf" event={"ID":"5c33e0e8-9379-4e3a-ab50-460a09d1efa8","Type":"ContainerDied","Data":"fb1496f76b2ee1b52d89715c674a0997ba2eb4d4f761868cc855589d751ae6c0"} Jul 12 15:49:31.903565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.903419 2563 scope.go:117] "RemoveContainer" containerID="f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852" Jul 12 15:49:31.905096 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.905071 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" event={"ID":"d17b6456-069d-4581-b14d-f6f6be888bc3","Type":"ContainerStarted","Data":"41efd0b7038b9893060738a54470cf5873812ef376761a6013ac01779e5ab31a"} Jul 12 15:49:31.912758 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.912673 2563 scope.go:117] "RemoveContainer" containerID="ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c" Jul 12 15:49:31.920865 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.920846 2563 scope.go:117] "RemoveContainer" containerID="f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852" Jul 12 15:49:31.921117 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:49:31.921100 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852\": container with ID starting with f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852 not found: ID does not exist" containerID="f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852" Jul 12 15:49:31.921164 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.921126 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852"} err="failed to get container status \"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852\": rpc error: code = NotFound desc = could not find container \"f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852\": container with ID starting with f6b568bbf788ea198e72a84c8a264ab60d36946acaad56f8af330f845123f852 not found: ID does not exist" Jul 12 15:49:31.921164 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.921146 2563 scope.go:117] "RemoveContainer" containerID="ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c" Jul 12 15:49:31.921380 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:49:31.921364 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c\": container with ID starting with ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c not found: ID does not exist" containerID="ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c" Jul 12 15:49:31.921452 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.921387 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c"} err="failed to get container status \"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c\": rpc error: code = NotFound desc = could not find container \"ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c\": container with ID starting with ed11faf4907695a3514214700153a7b392a056b3a64f8d029624de6c1ff5af4c not found: ID does not exist" Jul 12 15:49:31.925970 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.925945 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:31.931927 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:31.931902 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5d5046a9-openshift-default-758cf67859-s4lvf"] Jul 12 15:49:32.604808 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.604777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:32.688633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.688600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:32.754301 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.754271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:32.912218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.912186 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" event={"ID":"d17b6456-069d-4581-b14d-f6f6be888bc3","Type":"ContainerStarted","Data":"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9"} Jul 12 15:49:32.928578 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.928550 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" path="/var/lib/kubelet/pods/5c33e0e8-9379-4e3a-ab50-460a09d1efa8/volumes" Jul 12 15:49:32.939510 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.939455 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" podStartSLOduration=1.939439506 podStartE2EDuration="1.939439506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:49:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:49:32.938891397 +0000 UTC m=+1758.489092194" watchObservedRunningTime="2026-07-12 15:49:32.939439506 +0000 UTC m=+1758.489640278" Jul 12 15:49:32.947844 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:32.947813 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:33.337493 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.337384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:33.435016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.434979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:33.701189 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.701145 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:33.705711 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.705687 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:33.917063 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.917028 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:33.918116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:33.918099 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:34.797831 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:34.797781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:34.840901 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:34.840867 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:49:37.063165 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.063089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:37.234420 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.234361 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:37.234867 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.234835 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" podUID="d17b6456-069d-4581-b14d-f6f6be888bc3" containerName="istio-proxy" containerID="cri-o://07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9" gracePeriod=30 Jul 12 15:49:37.257358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.257325 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:37.356949 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.356860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:37.416900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:37.416859 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:38.807407 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:38.807360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:42.590486 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.590459 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:42.641888 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.641801 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:49:42.673442 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673408 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lrg2\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673455 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673474 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673491 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673532 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673556 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673602 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673586 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673631 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673658 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs\") pod \"d17b6456-069d-4581-b14d-f6f6be888bc3\" (UID: \"d17b6456-069d-4581-b14d-f6f6be888bc3\") " Jul 12 15:49:42.673893 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673775 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:42.674051 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.673896 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.674051 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.674000 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:49:42.674149 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.674112 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:42.674282 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.674240 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:42.674601 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.674512 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:42.675898 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.675870 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:49:42.675993 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.675970 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:42.676245 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.676229 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:49:42.676245 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.676228 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2" (OuterVolumeSpecName: "kube-api-access-8lrg2") pod "d17b6456-069d-4581-b14d-f6f6be888bc3" (UID: "d17b6456-069d-4581-b14d-f6f6be888bc3"). InnerVolumeSpecName "kube-api-access-8lrg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:42.774904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774866 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.774904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774900 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lrg2\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-kube-api-access-8lrg2\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774917 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774931 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774945 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774958 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17b6456-069d-4581-b14d-f6f6be888bc3-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774969 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.775114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.774980 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17b6456-069d-4581-b14d-f6f6be888bc3-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:49:42.916382 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.916356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:42.963037 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.963007 2563 generic.go:356] "Generic (PLEG): container finished" podID="d17b6456-069d-4581-b14d-f6f6be888bc3" containerID="07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9" exitCode=0 Jul 12 15:49:42.963201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.963053 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" Jul 12 15:49:42.963201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.963079 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" event={"ID":"d17b6456-069d-4581-b14d-f6f6be888bc3","Type":"ContainerDied","Data":"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9"} Jul 12 15:49:42.963201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.963126 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4" event={"ID":"d17b6456-069d-4581-b14d-f6f6be888bc3","Type":"ContainerDied","Data":"41efd0b7038b9893060738a54470cf5873812ef376761a6013ac01779e5ab31a"} Jul 12 15:49:42.963201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.963147 2563 scope.go:117] "RemoveContainer" containerID="07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9" Jul 12 15:49:42.971434 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.971415 2563 scope.go:117] "RemoveContainer" containerID="07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9" Jul 12 15:49:42.971684 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:49:42.971666 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9\": container with ID starting with 07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9 not found: ID does not exist" containerID="07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9" Jul 12 15:49:42.971756 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.971695 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9"} err="failed to get container status \"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9\": rpc error: code = NotFound desc = could not find container \"07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9\": container with ID starting with 07edcc260a9901d66434c0ed017fc79a30a2ec9e70e82b05a54224e22d6d49c9 not found: ID does not exist" Jul 12 15:49:42.981483 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.981458 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:42.984882 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.984853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:42.986712 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:42.986692 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7ed6f425-openshift-default-69c8ff9c7f-brkk4"] Jul 12 15:49:43.366648 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:43.366552 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:43.466325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:43.466269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:43.567062 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:43.567037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:44.883725 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:44.883693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:44.929569 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:44.929539 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17b6456-069d-4581-b14d-f6f6be888bc3" path="/var/lib/kubelet/pods/d17b6456-069d-4581-b14d-f6f6be888bc3/volumes" Jul 12 15:49:50.618848 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:50.618817 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:50.678674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:50.678639 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:50.760616 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:50.760581 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:50.859942 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:50.859905 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:50.959827 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:50.959799 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:49:51.314340 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:49:51.314254 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:14.914426 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:14.914383 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:50:14.918022 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:14.917998 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:50:17.257153 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:17.257121 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:23.112468 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:23.112428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:23.168440 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:23.168384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:23.238362 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:23.238328 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:23.338235 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:23.338204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:39.178651 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:39.178594 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:44.951372 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:44.951336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:45.013584 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:45.013550 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:45.095862 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:45.095830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:45.195595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:45.195562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:55.620633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:55.620598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:50:56.397197 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:50:56.397153 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:02.159150 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:02.159113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:02.280106 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:02.280060 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:02.381994 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:02.381962 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:02.479087 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:02.478969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:03.190669 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:03.190619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:03.282747 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:03.282705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:03.383034 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:03.382995 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:04.635777 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:04.635743 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:51:04.955197 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:04.955154 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:10.726807 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:10.726755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:10.857186 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:10.857143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:11.380555 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:11.380521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:11.732792 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:11.732743 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:51:17.196108 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.196066 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:17.298078 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.298038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:17.420494 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.420463 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:17.681192 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681138 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:17.681572 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681556 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:51:17.681625 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681576 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c33e0e8-9379-4e3a-ab50-460a09d1efa8" containerName="istio-proxy" Jul 12 15:51:17.681625 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681606 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17b6456-069d-4581-b14d-f6f6be888bc3" containerName="istio-proxy" Jul 12 15:51:17.681625 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681615 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17b6456-069d-4581-b14d-f6f6be888bc3" containerName="istio-proxy" Jul 12 15:51:17.681719 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.681692 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17b6456-069d-4581-b14d-f6f6be888bc3" containerName="istio-proxy" Jul 12 15:51:17.684866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.684847 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.687172 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.687153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4d006861-openshift-default-dockercfg-jb4sk\"" Jul 12 15:51:17.699878 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.699846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:17.703025 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.702989 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:17.802038 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.802003 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:17.814126 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814099 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tnjs\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814289 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.814527 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.814391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915494 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tnjs\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915573 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915610 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915663 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.915710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.915731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.916034 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.916056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.916156 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916349 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.916326 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.916613 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.916349 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.918144 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.918110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.918422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.918382 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.923359 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.923337 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.923595 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.923577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tnjs\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs\") pod \"e2e-mt-4d006861-openshift-default-947488b47-qsnzv\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:17.994762 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:17.994671 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:18.127251 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.127154 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:18.129627 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:51:18.129599 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f51e3f1_7efb_458d_9346_44878f4238c0.slice/crio-809cd01e2e30345cee79e522843869b876145863944b81ebda87ccf4b32ae738 WatchSource:0}: Error finding container 809cd01e2e30345cee79e522843869b876145863944b81ebda87ccf4b32ae738: Status 404 returned error can't find the container with id 809cd01e2e30345cee79e522843869b876145863944b81ebda87ccf4b32ae738 Jul 12 15:51:18.132052 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.132016 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:18.132159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.132118 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:18.132225 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.132167 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:18.353887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.353791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerStarted","Data":"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513"} Jul 12 15:51:18.353887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.353854 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerStarted","Data":"809cd01e2e30345cee79e522843869b876145863944b81ebda87ccf4b32ae738"} Jul 12 15:51:18.374591 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.374531 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" podStartSLOduration=1.374508268 podStartE2EDuration="1.374508268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:51:18.373604805 +0000 UTC m=+1863.923805617" watchObservedRunningTime="2026-07-12 15:51:18.374508268 +0000 UTC m=+1863.924709044" Jul 12 15:51:18.995749 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:18.995720 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:20.000431 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.000385 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:20.343433 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.343326 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:51:20.362680 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.362655 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:20.363707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.363691 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:20.509348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.509313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:20.640278 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.640188 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:20.712958 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.712921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:20.854479 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:20.854445 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:21.178016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:21.177985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:21.275523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:21.275482 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:24.920078 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:24.920044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:25.553710 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:25.553676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:25.653448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:25.653418 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:25.985631 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:25.985601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:29.641060 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:29.641018 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:51:31.658157 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:31.658099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:31.725255 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:31.725217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:32.037634 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:32.037502 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:32.290325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:32.290203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:32.427449 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:32.427378 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:34.421348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.421314 2563 generic.go:356] "Generic (PLEG): container finished" podID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerID="5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513" exitCode=0 Jul 12 15:51:34.421348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.421353 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerDied","Data":"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513"} Jul 12 15:51:34.421815 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.421706 2563 scope.go:117] "RemoveContainer" containerID="5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513" Jul 12 15:51:34.422661 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.422627 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:34.422786 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.422735 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:34.422856 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:34.422788 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:35.362866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:35.362829 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:35.426851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:35.426814 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerStarted","Data":"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247"} Jul 12 15:51:35.995279 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:35.995239 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:35.999691 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:35.999661 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:36.430979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:36.430953 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:36.432012 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:36.431991 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:38.199864 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.199836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:38.300728 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.300693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:38.419493 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.419462 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:38.434706 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.434676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:38.438919 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.438896 2563 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" secret="" err="secret \"e2e-mt-4d006861-openshift-default-dockercfg-jb4sk\" not found" Jul 12 15:51:38.680116 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.680076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:38.823024 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.822993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:38.922496 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:38.922460 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:39.025975 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:39.025898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:39.443158 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:39.443114 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" containerID="cri-o://53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247" gracePeriod=30 Jul 12 15:51:40.438963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:40.438925 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:51:44.692809 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.692783 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:44.740639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740566 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740602 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740634 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740649 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740666 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740688 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740727 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740752 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740841 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tnjs\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs\") pod \"4f51e3f1-7efb-458d-9346-44878f4238c0\" (UID: \"4f51e3f1-7efb-458d-9346-44878f4238c0\") " Jul 12 15:51:44.740906 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740892 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:51:44.741268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.740984 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:51:44.741268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.741115 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.741268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.741132 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.741609 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.741287 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:51:44.741609 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.741434 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:51:44.742004 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.741972 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:51:44.743058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.743019 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:51:44.743158 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.743062 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:51:44.743158 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.743087 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:51:44.743233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.743194 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs" (OuterVolumeSpecName: "kube-api-access-9tnjs") pod "4f51e3f1-7efb-458d-9346-44878f4238c0" (UID: "4f51e3f1-7efb-458d-9346-44878f4238c0"). InnerVolumeSpecName "kube-api-access-9tnjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:51:44.751975 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.751946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:44.842296 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842266 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842296 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842295 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842296 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842304 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842314 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842322 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f51e3f1-7efb-458d-9346-44878f4238c0-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842331 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tnjs\" (UniqueName: \"kubernetes.io/projected/4f51e3f1-7efb-458d-9346-44878f4238c0-kube-api-access-9tnjs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.842548 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.842340 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f51e3f1-7efb-458d-9346-44878f4238c0-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:51:44.858111 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.858087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:44.952842 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:44.952810 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:45.087272 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.087179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:45.469230 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.469190 2563 generic.go:356] "Generic (PLEG): container finished" podID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerID="53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247" exitCode=0 Jul 12 15:51:45.469488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.469251 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" Jul 12 15:51:45.469488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.469275 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerDied","Data":"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247"} Jul 12 15:51:45.469488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.469330 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv" event={"ID":"4f51e3f1-7efb-458d-9346-44878f4238c0","Type":"ContainerDied","Data":"809cd01e2e30345cee79e522843869b876145863944b81ebda87ccf4b32ae738"} Jul 12 15:51:45.469488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.469363 2563 scope.go:117] "RemoveContainer" containerID="53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247" Jul 12 15:51:45.477605 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.477587 2563 scope.go:117] "RemoveContainer" containerID="5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513" Jul 12 15:51:45.485157 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.485135 2563 scope.go:117] "RemoveContainer" containerID="53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247" Jul 12 15:51:45.485466 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:51:45.485439 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247\": container with ID starting with 53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247 not found: ID does not exist" containerID="53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247" Jul 12 15:51:45.485580 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.485476 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247"} err="failed to get container status \"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247\": rpc error: code = NotFound desc = could not find container \"53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247\": container with ID starting with 53546b2ba478d375c155231447bcdd9d0887209112e2e168797f931008bc0247 not found: ID does not exist" Jul 12 15:51:45.485580 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.485494 2563 scope.go:117] "RemoveContainer" containerID="5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513" Jul 12 15:51:45.485780 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:51:45.485761 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513\": container with ID starting with 5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513 not found: ID does not exist" containerID="5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513" Jul 12 15:51:45.485842 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.485787 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513"} err="failed to get container status \"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513\": rpc error: code = NotFound desc = could not find container \"5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513\": container with ID starting with 5e80a74f5047d12ca59dc59ef9425f7715ad4d75218ed17dc6f1fcc8279b3513 not found: ID does not exist" Jul 12 15:51:45.492829 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.492807 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:45.496282 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:45.496262 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4d006861-openshift-default-947488b47-qsnzv"] Jul 12 15:51:46.929281 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:46.929239 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" path="/var/lib/kubelet/pods/4f51e3f1-7efb-458d-9346-44878f4238c0/volumes" Jul 12 15:51:50.673183 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:50.673143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:50.733159 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:50.733120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:57.367029 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.366992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:57.621619 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621546 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:51:57.621821 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621809 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.621861 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621822 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.621861 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621836 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.621861 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621843 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.621952 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621892 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.621952 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.621900 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f51e3f1-7efb-458d-9346-44878f4238c0" containerName="istio-proxy" Jul 12 15:51:57.625077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.625055 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.627571 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.627542 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cd9517a2-openshift-default-dockercfg-gltdc\"" Jul 12 15:51:57.637728 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.637697 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:51:57.648348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.648322 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:57.744049 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744115 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744141 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744175 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744232 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744456 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.744456 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.744328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckbfl\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845020 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.844980 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845020 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845167 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845248 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845528 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845528 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845337 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckbfl\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845528 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845528 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845466 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845528 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845603 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.845999 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.845977 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.846170 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.846152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.847660 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.847642 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.847756 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.847720 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.853133 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.853111 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.853963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.853941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckbfl\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl\") pod \"e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:57.935901 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:57.935862 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:58.016564 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.016535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:51:58.080845 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.080817 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:51:58.083718 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:51:58.083682 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd25a0c1_ee0c_48ee_bcdd_4c2d5844f928.slice/crio-7a9e7b2347356ee07677a31bcdcf565dae1c50aefcf1c2325b68fc6aa682d881 WatchSource:0}: Error finding container 7a9e7b2347356ee07677a31bcdcf565dae1c50aefcf1c2325b68fc6aa682d881: Status 404 returned error can't find the container with id 7a9e7b2347356ee07677a31bcdcf565dae1c50aefcf1c2325b68fc6aa682d881 Jul 12 15:51:58.086101 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.086066 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:58.086221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.086149 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:58.086221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.086193 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:51:58.525703 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.525666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerStarted","Data":"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71"} Jul 12 15:51:58.526211 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.525716 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerStarted","Data":"7a9e7b2347356ee07677a31bcdcf565dae1c50aefcf1c2325b68fc6aa682d881"} Jul 12 15:51:58.543692 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.543629 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podStartSLOduration=1.5436048150000001 podStartE2EDuration="1.543604815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:51:58.542528431 +0000 UTC m=+1904.092729232" watchObservedRunningTime="2026-07-12 15:51:58.543604815 +0000 UTC m=+1904.093805591" Jul 12 15:51:58.936272 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:58.936244 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:51:59.940866 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:51:59.940829 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:00.524825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.524777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:00.534291 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.534258 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:00.535164 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.535146 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:00.605561 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.605504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:00.663234 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.663195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:00.879763 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:00.879674 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:01.038532 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:01.038499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:01.138887 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:01.138805 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:05.010355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:05.010329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:07.059875 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:07.059831 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:52:13.412239 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:13.412197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:15.536107 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.536072 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 12 15:52:15.536550 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.536137 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 12 15:52:15.600930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.600898 2563 generic.go:356] "Generic (PLEG): container finished" podID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerID="9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71" exitCode=0 Jul 12 15:52:15.601059 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.600975 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerDied","Data":"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71"} Jul 12 15:52:15.601366 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.601353 2563 scope.go:117] "RemoveContainer" containerID="9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71" Jul 12 15:52:15.602341 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.602313 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:52:15.602474 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.602415 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:52:15.602539 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:15.602470 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:52:16.607233 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:16.607194 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerStarted","Data":"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590"} Jul 12 15:52:16.936677 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:16.936642 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:16.936869 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:16.936844 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 12 15:52:16.936919 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:16.936891 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 12 15:52:17.550751 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:17.550721 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:52:17.936888 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:17.936845 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 12 15:52:17.937385 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:17.936913 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 12 15:52:18.937003 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:18.936963 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 12 15:52:18.937534 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:18.937046 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 12 15:52:19.936828 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:19.936784 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 12 15:52:19.937008 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:19.936858 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 12 15:52:20.940322 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:20.940289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:20.940813 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:20.940585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:20.941307 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:20.941288 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:52:29.276259 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:29.276225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:29.377686 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:29.377650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:29.647673 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:29.647588 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:52:30.427961 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:30.427924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:30.529136 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:30.529104 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:32.202385 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:32.202343 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:32.582259 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:32.582163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:32.699155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:32.699121 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:32.909674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:32.909640 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:38.268153 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:38.268123 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:43.288125 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:43.288087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:52:43.541202 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:52:43.541121 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:53:19.520746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:19.520711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:20.338126 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:20.338088 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:22.543514 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:22.543478 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:53:25.736900 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:25.736865 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:31.241589 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:31.241547 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:31.577455 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:31.577336 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:53:31.577627 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:31.577585 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" containerID="cri-o://81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590" gracePeriod=30 Jul 12 15:53:32.318098 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:32.318062 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:32.807589 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:32.807550 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:32.906099 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:32.906070 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:33.003278 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:33.003250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:33.682337 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:33.682303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:33.744519 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:33.744484 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:33.804369 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:33.804329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:35.192204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:35.192164 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:35.943265 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:35.943232 2563 patch_prober.go:28] interesting pod/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 15:53:35.943461 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:35.943287 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 15:53:36.190613 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.190580 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:53:36.824367 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.824341 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:53:36.857685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857606 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857650 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857685 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857676 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckbfl\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857722 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857769 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857791 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857822 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857853 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.857930 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.857871 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket\") pod \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\" (UID: \"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928\") " Jul 12 15:53:36.858227 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.858093 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:53:36.858291 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.858277 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:53:36.858627 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.858458 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:53:36.858627 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.858478 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:53:36.859384 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.859357 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:53:36.860325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.860299 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl" (OuterVolumeSpecName: "kube-api-access-ckbfl") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "kube-api-access-ckbfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:53:36.860468 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.860420 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:53:36.860639 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.860613 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:53:36.860831 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.860806 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" (UID: "dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:53:36.954416 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.954369 2563 generic.go:356] "Generic (PLEG): container finished" podID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerID="81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590" exitCode=0 Jul 12 15:53:36.954596 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.954445 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" Jul 12 15:53:36.954596 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.954452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerDied","Data":"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590"} Jul 12 15:53:36.954596 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.954509 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x" event={"ID":"dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928","Type":"ContainerDied","Data":"7a9e7b2347356ee07677a31bcdcf565dae1c50aefcf1c2325b68fc6aa682d881"} Jul 12 15:53:36.954596 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.954533 2563 scope.go:117] "RemoveContainer" containerID="81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590" Jul 12 15:53:36.958609 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958586 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958609 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958608 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958617 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckbfl\" (UniqueName: \"kubernetes.io/projected/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-kube-api-access-ckbfl\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958628 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958636 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958645 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958652 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958659 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.958757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.958667 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:53:36.963523 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.963508 2563 scope.go:117] "RemoveContainer" containerID="9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71" Jul 12 15:53:36.971260 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.971229 2563 scope.go:117] "RemoveContainer" containerID="81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590" Jul 12 15:53:36.971764 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:53:36.971734 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590\": container with ID starting with 81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590 not found: ID does not exist" containerID="81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590" Jul 12 15:53:36.971869 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.971777 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590"} err="failed to get container status \"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590\": rpc error: code = NotFound desc = could not find container \"81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590\": container with ID starting with 81c1e7dfe5109d0f91256a8fad8635a45172306b926c2024d4d7dde083011590 not found: ID does not exist" Jul 12 15:53:36.971869 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.971802 2563 scope.go:117] "RemoveContainer" containerID="9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71" Jul 12 15:53:36.972147 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:53:36.972091 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71\": container with ID starting with 9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71 not found: ID does not exist" containerID="9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71" Jul 12 15:53:36.972147 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.972118 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71"} err="failed to get container status \"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71\": rpc error: code = NotFound desc = could not find container \"9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71\": container with ID starting with 9e205f1c68d7a9c5dbed40956d52b33f19a24770e55c14d5805e546aaf0d5b71 not found: ID does not exist" Jul 12 15:53:36.973295 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.973277 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:53:36.976321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:36.976301 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cd9517a2-openshift-default-6d466d4b95-zl75x"] Jul 12 15:53:38.929376 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:38.929339 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" path="/var/lib/kubelet/pods/dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928/volumes" Jul 12 15:53:41.027106 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:41.027071 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:41.087688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:41.087656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:41.688902 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:41.688866 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:47.378939 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:47.378903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:47.434696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:47.434658 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:47.505175 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:47.505141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:47.599575 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:47.599543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:48.542875 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:48.542846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:53:58.044488 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:58.044451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:53:58.388205 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:53:58.388130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:19.312834 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:19.312797 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:24.934963 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:24.934933 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:25.036875 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.036837 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:25.277975 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.277884 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:25.544321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.544240 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:54:25.544529 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.544514 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:25.544611 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.544534 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:25.544611 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.544610 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:25.544727 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.544620 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:25.547709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.547690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.550016 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.549998 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8ac237-openshift-default-dockercfg-qr29t\"" Jul 12 15:54:25.557586 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.557563 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:54:25.568435 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.568408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:25.641706 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641706 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641738 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641811 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.641904 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.642068 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641905 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.642068 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.641923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxzbx\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.665481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.665444 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:25.742723 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742723 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742726 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742789 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742833 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742882 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.742979 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.742909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxzbx\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.743298 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.743128 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.743355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.743315 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.743355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.743341 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.743490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.743421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.743596 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.743577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.745533 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.745510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.745675 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.745654 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.753135 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.753111 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.753238 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.753221 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxzbx\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx\") pod \"e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.858179 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.858077 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:25.990997 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.990969 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:54:25.993441 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:54:25.993390 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad63898_e4e9_4be6_8893_f0ec0436611d.slice/crio-dbac38b68d73f5126128950d5b44a3c3c2f19068a93dd71c6711f219c0b9cfdb WatchSource:0}: Error finding container dbac38b68d73f5126128950d5b44a3c3c2f19068a93dd71c6711f219c0b9cfdb: Status 404 returned error can't find the container with id dbac38b68d73f5126128950d5b44a3c3c2f19068a93dd71c6711f219c0b9cfdb Jul 12 15:54:25.995506 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.995474 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:25.995569 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.995543 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:25.995606 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:25.995572 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:26.161535 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:26.161495 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerStarted","Data":"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5"} Jul 12 15:54:26.161716 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:26.161573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerStarted","Data":"dbac38b68d73f5126128950d5b44a3c3c2f19068a93dd71c6711f219c0b9cfdb"} Jul 12 15:54:26.180713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:26.180666 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podStartSLOduration=1.180648248 podStartE2EDuration="1.180648248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:54:26.178996165 +0000 UTC m=+2051.729196956" watchObservedRunningTime="2026-07-12 15:54:26.180648248 +0000 UTC m=+2051.730849023" Jul 12 15:54:26.859058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:26.859016 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:27.863204 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:27.863171 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:28.170230 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.170194 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:28.171139 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.171116 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:28.175200 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.175176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:28.281451 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.281414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:28.427059 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.426957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:28.863923 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.863832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:28.965496 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:28.965465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:29.065411 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:29.065373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:32.658077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:32.658033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:32.979687 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:32.979593 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:33.235357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.235255 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:54:33.235688 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.235668 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:33.235769 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.235691 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd25a0c1-ee0c-48ee-bcdd-4c2d5844f928" containerName="istio-proxy" Jul 12 15:54:33.238977 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.238954 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.241577 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.241553 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0e354c-openshift-default-dockercfg-r8vtd\"" Jul 12 15:54:33.249183 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.249159 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:54:33.260977 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.260953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:33.304497 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304653 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg2nm\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304653 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304653 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304653 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.304863 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.304751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405638 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405453 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405638 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405638 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405539 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg2nm\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405638 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405638 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.405938 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.406007 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405894 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406077 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.405862 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406293 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.406247 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406293 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.406278 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.406483 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.406311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.407985 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.407962 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.408306 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.408282 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.413358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.413331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.413477 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.413362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg2nm\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm\") pod \"e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.549833 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.549728 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:33.623592 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.623561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:33.690943 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.690878 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:54:33.693229 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:54:33.693203 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf38b36cd_d082_48cf_a7e9_b9494ecdf5d8.slice/crio-20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5 WatchSource:0}: Error finding container 20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5: Status 404 returned error can't find the container with id 20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5 Jul 12 15:54:33.695445 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.695386 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:33.695536 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.695481 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:33.695536 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:33.695510 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:34.197558 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:34.197527 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerStarted","Data":"9985c4cf3c657abc30d6d3588899e761938ffe060f4def72b21764f412caa946"} Jul 12 15:54:34.197725 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:34.197574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerStarted","Data":"20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5"} Jul 12 15:54:34.218352 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:34.218295 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podStartSLOduration=1.218279711 podStartE2EDuration="1.218279711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:54:34.216390139 +0000 UTC m=+2059.766590934" watchObservedRunningTime="2026-07-12 15:54:34.218279711 +0000 UTC m=+2059.768480485" Jul 12 15:54:34.550564 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:34.550465 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:34.555426 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:34.555377 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:35.202118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:35.202089 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:35.202990 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:35.202967 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:35.836717 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:35.836681 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:54:36.188108 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.188071 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:36.270919 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.270885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:36.342920 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.342880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:36.530095 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.530006 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:36.725889 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.725860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:36.798176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:36.798083 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:40.687677 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:40.687642 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:43.492665 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:43.492627 2563 patch_prober.go:28] interesting pod/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": read tcp 10.132.0.2:59784->10.132.0.28:15021: read: connection reset by peer" start-of-body= Jul 12 15:54:43.493091 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:43.492712 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": read tcp 10.132.0.2:59784->10.132.0.28:15021: read: connection reset by peer" Jul 12 15:54:44.246960 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.246927 2563 generic.go:356] "Generic (PLEG): container finished" podID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerID="2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5" exitCode=0 Jul 12 15:54:44.247148 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.247001 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerDied","Data":"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5"} Jul 12 15:54:44.247439 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.247425 2563 scope.go:117] "RemoveContainer" containerID="2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5" Jul 12 15:54:44.248373 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.248346 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:44.248447 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.248429 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:44.248490 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:44.248461 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:45.266794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:45.266751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerStarted","Data":"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8"} Jul 12 15:54:45.858636 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:45.858593 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:45.858858 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:45.858728 2563 patch_prober.go:28] interesting pod/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 12 15:54:45.858858 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:45.858774 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 12 15:54:45.939524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:45.939461 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:54:46.705058 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:46.705004 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:46.858891 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:46.858842 2563 patch_prober.go:28] interesting pod/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 12 15:54:46.859098 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:46.858906 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 12 15:54:47.859195 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:47.859152 2563 patch_prober.go:28] interesting pod/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 12 15:54:47.859633 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:47.859213 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 12 15:54:48.858714 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:48.858667 2563 patch_prober.go:28] interesting pod/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 12 15:54:48.858892 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:48.858743 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 12 15:54:49.862895 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:49.862858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:49.863355 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:49.863284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:49.864025 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:49.864008 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:54:50.203465 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.203427 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:50.203663 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.203486 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:50.245802 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.245748 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:50.291802 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.291711 2563 generic.go:356] "Generic (PLEG): container finished" podID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerID="9985c4cf3c657abc30d6d3588899e761938ffe060f4def72b21764f412caa946" exitCode=0 Jul 12 15:54:50.291802 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.291790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerDied","Data":"9985c4cf3c657abc30d6d3588899e761938ffe060f4def72b21764f412caa946"} Jul 12 15:54:50.292268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.292252 2563 scope.go:117] "RemoveContainer" containerID="9985c4cf3c657abc30d6d3588899e761938ffe060f4def72b21764f412caa946" Jul 12 15:54:50.293161 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.293126 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:50.293215 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.293201 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:50.293254 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.293241 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:54:50.317075 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:50.317042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:51.298585 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:51.298548 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerStarted","Data":"ab13bcd71c4e53ef373b5e3b28bcd81a42ac353e1e4a432ac2cee9bd89aaab5f"} Jul 12 15:54:51.550260 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:51.550155 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:51.550476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:51.550451 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:51.550563 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:51.550509 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:52.550870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:52.550826 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:52.551338 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:52.550889 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:53.550585 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:53.550546 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:53.550760 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:53.550610 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:54.551108 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:54.551073 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:54.551508 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:54.551129 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:55.143419 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:55.143355 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:54:55.550848 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:55.550808 2563 patch_prober.go:28] interesting pod/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 12 15:54:55.551033 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:55.550871 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 12 15:54:56.554437 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:56.554378 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:56.554910 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:56.554799 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:56.555492 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:56.555472 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:54:57.331292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:57.331256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:54:57.432723 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:54:57.432693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:05.235761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:05.235729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:10.041694 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:10.041657 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:55:14.941672 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:14.941644 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:55:14.944970 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:14.944946 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 15:55:39.038362 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:39.038328 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:40.013598 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:40.013562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:40.163877 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:40.163839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:40.527905 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:40.527860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:40.629364 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:40.629331 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:40.746256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:40.746217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:55:43.242290 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:43.242250 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:55:45.632515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:55:45.632474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:01.178593 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:01.178560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:22.414706 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:22.414670 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:22.704700 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:22.704597 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:56:22.704989 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:22.704955 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" containerID="cri-o://171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8" gracePeriod=30 Jul 12 15:56:22.733579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:22.733530 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:22.940541 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:22.940508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:23.103218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:23.103139 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:23.523287 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:23.523256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:23.623476 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:23.623441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:23.723383 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:23.723350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:23.783857 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:23.783773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:26.387770 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:26.387736 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:26.934033 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:26.933996 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:56:27.950154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.950131 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:56:27.961153 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961133 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxzbx\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961269 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961170 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961269 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961228 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961269 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961259 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961291 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961328 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961354 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961391 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961457 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961452 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo\") pod \"7ad63898-e4e9-4be6-8893-f0ec0436611d\" (UID: \"7ad63898-e4e9-4be6-8893-f0ec0436611d\") " Jul 12 15:56:27.961702 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961550 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:27.961702 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961532 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:27.961811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961754 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:27.961811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961776 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:27.961811 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961781 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:27.962007 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.961881 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:56:27.962194 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.962110 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:27.963656 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.963622 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:56:27.963765 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.963747 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:27.964017 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.963994 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx" (OuterVolumeSpecName: "kube-api-access-vxzbx") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "kube-api-access-vxzbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:27.964109 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:27.964030 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ad63898-e4e9-4be6-8893-f0ec0436611d" (UID: "7ad63898-e4e9-4be6-8893-f0ec0436611d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:28.063114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063077 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxzbx\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-kube-api-access-vxzbx\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063109 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063114 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063118 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063126 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad63898-e4e9-4be6-8893-f0ec0436611d-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063135 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063142 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad63898-e4e9-4be6-8893-f0ec0436611d-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.063422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.063149 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad63898-e4e9-4be6-8893-f0ec0436611d-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:28.728695 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.728653 2563 generic.go:356] "Generic (PLEG): container finished" podID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerID="171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8" exitCode=0 Jul 12 15:56:28.728870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.728708 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" Jul 12 15:56:28.728870 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.728705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerDied","Data":"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8"} Jul 12 15:56:28.728940 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.728864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp" event={"ID":"7ad63898-e4e9-4be6-8893-f0ec0436611d","Type":"ContainerDied","Data":"dbac38b68d73f5126128950d5b44a3c3c2f19068a93dd71c6711f219c0b9cfdb"} Jul 12 15:56:28.728940 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.728902 2563 scope.go:117] "RemoveContainer" containerID="171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8" Jul 12 15:56:28.737749 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.737727 2563 scope.go:117] "RemoveContainer" containerID="2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5" Jul 12 15:56:28.745484 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.745463 2563 scope.go:117] "RemoveContainer" containerID="171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8" Jul 12 15:56:28.745738 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:56:28.745717 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8\": container with ID starting with 171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8 not found: ID does not exist" containerID="171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8" Jul 12 15:56:28.745806 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.745746 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8"} err="failed to get container status \"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8\": rpc error: code = NotFound desc = could not find container \"171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8\": container with ID starting with 171444e603e0a96c7f23556f8e85c3781dfe6b19381f0aada97c538b93d12bc8 not found: ID does not exist" Jul 12 15:56:28.745806 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.745775 2563 scope.go:117] "RemoveContainer" containerID="2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5" Jul 12 15:56:28.746003 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:56:28.745977 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5\": container with ID starting with 2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5 not found: ID does not exist" containerID="2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5" Jul 12 15:56:28.746104 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.746005 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5"} err="failed to get container status \"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5\": rpc error: code = NotFound desc = could not find container \"2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5\": container with ID starting with 2cee01995b3fa927e5e6589c0a7c208688e9ca09f2760b9a6c145d5748262fa5 not found: ID does not exist" Jul 12 15:56:28.749570 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.749548 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:56:28.753498 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.753479 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8ac237-openshift-default-7d96669fdf-v74mp"] Jul 12 15:56:28.929221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:28.929179 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" path="/var/lib/kubelet/pods/7ad63898-e4e9-4be6-8893-f0ec0436611d/volumes" Jul 12 15:56:29.285110 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.285079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:29.385373 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.385342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:29.520151 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.520121 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:56:29.520375 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.520353 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" containerID="cri-o://ab13bcd71c4e53ef373b5e3b28bcd81a42ac353e1e4a432ac2cee9bd89aaab5f" gracePeriod=30 Jul 12 15:56:29.535471 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.535360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:29.760894 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:29.760858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:30.091868 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.091836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:30.368419 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368317 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368621 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368634 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368659 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368665 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368709 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.368753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.368715 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad63898-e4e9-4be6-8893-f0ec0436611d" containerName="istio-proxy" Jul 12 15:56:30.373075 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.373058 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.375647 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.375623 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a5af97-openshift-default-dockercfg-fzclp\"" Jul 12 15:56:30.382599 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.382579 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:56:30.397314 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.397281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:30.483148 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483117 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483160 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvsnx\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483461 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.483607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.483538 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584578 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584636 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvsnx\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584735 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.584757 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585110 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.584782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585110 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.585034 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585253 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.585229 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585318 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.585291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.585360 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.585686 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.585659 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.587049 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.587030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.587322 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.587304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.592994 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.592972 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.593358 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.593338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvsnx\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx\") pod \"e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.680121 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.680085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:30.684095 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.684072 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:30.831056 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.831031 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:56:30.832966 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:56:30.832940 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f16af0b_d1b5_4611_8f5c_c492bd93f9e0.slice/crio-3287b9e334de8c27c9c66e15f98a1dd456f2a2062cf0bcd6c80ad7774e5e50c7 WatchSource:0}: Error finding container 3287b9e334de8c27c9c66e15f98a1dd456f2a2062cf0bcd6c80ad7774e5e50c7: Status 404 returned error can't find the container with id 3287b9e334de8c27c9c66e15f98a1dd456f2a2062cf0bcd6c80ad7774e5e50c7 Jul 12 15:56:30.835068 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.835036 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:30.835151 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.835098 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:30.835151 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:30.835127 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:31.743804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:31.743773 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerStarted","Data":"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93"} Jul 12 15:56:31.744220 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:31.743822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerStarted","Data":"3287b9e334de8c27c9c66e15f98a1dd456f2a2062cf0bcd6c80ad7774e5e50c7"} Jul 12 15:56:31.764158 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:31.764112 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podStartSLOduration=1.764096464 podStartE2EDuration="1.764096464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:56:31.762846029 +0000 UTC m=+2177.313046822" watchObservedRunningTime="2026-07-12 15:56:31.764096464 +0000 UTC m=+2177.314297240" Jul 12 15:56:32.685104 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:32.685064 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:32.690042 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:32.690021 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:32.748065 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:32.748029 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:32.749051 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:32.749032 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:33.789115 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:33.789073 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:33.912203 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:33.912174 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:34.005001 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.004967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:34.105035 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.104949 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:34.415947 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.415918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:34.514549 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.514518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:34.758649 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.758621 2563 generic.go:356] "Generic (PLEG): container finished" podID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerID="ab13bcd71c4e53ef373b5e3b28bcd81a42ac353e1e4a432ac2cee9bd89aaab5f" exitCode=0 Jul 12 15:56:34.758813 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.758694 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerDied","Data":"ab13bcd71c4e53ef373b5e3b28bcd81a42ac353e1e4a432ac2cee9bd89aaab5f"} Jul 12 15:56:34.758813 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.758760 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" event={"ID":"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8","Type":"ContainerDied","Data":"20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5"} Jul 12 15:56:34.758813 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.758780 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20ce7364a0dbba2e160e7deec54e22b2cc968aab0d67c9a56c65c21b76d8f6a5" Jul 12 15:56:34.758813 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.758794 2563 scope.go:117] "RemoveContainer" containerID="9985c4cf3c657abc30d6d3588899e761938ffe060f4def72b21764f412caa946" Jul 12 15:56:34.770091 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.770075 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:56:34.818328 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818300 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818328 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818339 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818368 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg2nm\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818440 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818465 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818501 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818539 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818564 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818597 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token\") pod \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\" (UID: \"f38b36cd-d082-48cf-a7e9-b9494ecdf5d8\") " Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818755 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:56:34.818826 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818753 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:34.819309 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.818991 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.819309 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.819012 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.819309 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.819134 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:34.819526 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.819501 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:34.819696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.819674 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:34.820716 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.820691 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:56:34.820940 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.820918 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:34.821181 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.821165 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm" (OuterVolumeSpecName: "kube-api-access-qg2nm") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "kube-api-access-qg2nm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:34.821409 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.821374 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" (UID: "f38b36cd-d082-48cf-a7e9-b9494ecdf5d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:56:34.920047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920012 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920042 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg2nm\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-kube-api-access-qg2nm\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920051 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920061 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920071 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920079 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:34.920325 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:34.920087 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:56:35.766005 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:35.765968 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv" Jul 12 15:56:35.800926 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:35.800892 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:56:35.806645 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:35.806620 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0e354c-openshift-default-678cc47c47-2xxbv"] Jul 12 15:56:36.433848 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:36.433812 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:56:36.928967 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:36.928938 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" path="/var/lib/kubelet/pods/f38b36cd-d082-48cf-a7e9-b9494ecdf5d8/volumes" Jul 12 15:56:38.248832 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.248796 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:38.681696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.681660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:38.938500 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.938392 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:56:38.939371 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.939345 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.940478 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.940456 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.940641 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.940619 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.940732 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.940642 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.940794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.940746 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.940794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.940760 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f38b36cd-d082-48cf-a7e9-b9494ecdf5d8" containerName="istio-proxy" Jul 12 15:56:38.945149 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.945128 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:38.947744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.947725 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e6ad77-openshift-default-dockercfg-thmlk\"" Jul 12 15:56:38.951771 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.951748 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:56:38.968192 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:38.968152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:39.054330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054452 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054524 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054708 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054708 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ldcl\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054708 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.054843 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.054740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.155947 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.155903 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.155947 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.155946 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.155967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156221 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156374 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156374 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156374 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ldcl\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156374 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156628 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156576 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156746 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156881 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156733 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.156881 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156800 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.157024 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.156999 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.158485 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.158465 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.158696 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.158679 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.165345 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.165320 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.165605 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.165587 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ldcl\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl\") pod \"e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.258616 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.258521 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:39.381506 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.381423 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:39.432485 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.432379 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:56:39.440604 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:56:39.440568 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f42c849_5e09_4b49_86dc_bd64a71a3214.slice/crio-f36d1eef9021be2b0d6e926fcdbd1b7d6bd1c5c0f18ea2af74dc9a4c70f6a78b WatchSource:0}: Error finding container f36d1eef9021be2b0d6e926fcdbd1b7d6bd1c5c0f18ea2af74dc9a4c70f6a78b: Status 404 returned error can't find the container with id f36d1eef9021be2b0d6e926fcdbd1b7d6bd1c5c0f18ea2af74dc9a4c70f6a78b Jul 12 15:56:39.442693 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.442660 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:39.442791 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.442748 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:39.442791 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.442780 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:39.785701 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.785658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerStarted","Data":"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1"} Jul 12 15:56:39.785881 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.785725 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerStarted","Data":"f36d1eef9021be2b0d6e926fcdbd1b7d6bd1c5c0f18ea2af74dc9a4c70f6a78b"} Jul 12 15:56:39.805139 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:39.805085 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podStartSLOduration=1.805067535 podStartE2EDuration="1.805067535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:56:39.802880637 +0000 UTC m=+2185.353081434" watchObservedRunningTime="2026-07-12 15:56:39.805067535 +0000 UTC m=+2185.355268310" Jul 12 15:56:40.259202 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:40.259160 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:40.264283 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:40.264258 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:40.790758 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:40.790729 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:40.791660 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:40.791643 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:41.804053 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:41.803969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:41.934234 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:41.934203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:42.016047 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:42.016002 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:42.516141 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:42.516107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:42.619783 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:42.619749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:44.811184 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.811148 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerID="556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93" exitCode=0 Jul 12 15:56:44.811654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.811220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerDied","Data":"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93"} Jul 12 15:56:44.811654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.811636 2563 scope.go:117] "RemoveContainer" containerID="556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93" Jul 12 15:56:44.812628 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.812602 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:44.812700 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.812666 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:44.812739 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:44.812696 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:45.821783 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:45.821746 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerStarted","Data":"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9"} Jul 12 15:56:46.266411 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:46.266370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:46.684565 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:46.684533 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:46.685671 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:46.685640 2563 patch_prober.go:28] interesting pod/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 12 15:56:46.685801 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:46.685692 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 12 15:56:47.684709 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:47.684671 2563 patch_prober.go:28] interesting pod/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 12 15:56:47.685093 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:47.684779 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 12 15:56:47.748901 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:47.748865 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:48.684804 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:48.684768 2563 patch_prober.go:28] interesting pod/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 12 15:56:48.685179 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:48.684819 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 12 15:56:49.041654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:49.041552 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:56:49.688518 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:49.688487 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:49.689687 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:49.689662 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:56:54.269773 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:54.269734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:54.646390 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:54.646312 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:54.747055 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:54.747007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:54.887823 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:54.887788 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:56:56.832579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:56.832537 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 15:56:56.832968 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:56.832598 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 15:56:57.880773 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.880736 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerID="21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1" exitCode=0 Jul 12 15:56:57.881188 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.880806 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerDied","Data":"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1"} Jul 12 15:56:57.881188 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.881177 2563 scope.go:117] "RemoveContainer" containerID="21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1" Jul 12 15:56:57.882157 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.882132 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:57.882208 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.882200 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:57.882242 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:57.882233 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:56:58.832223 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:58.832185 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:56:58.887246 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:58.887213 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerStarted","Data":"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351"} Jul 12 15:56:59.259300 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:59.259257 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:56:59.259534 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:59.259512 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 12 15:56:59.259583 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:56:59.259565 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 12 15:57:00.259600 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.259561 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 12 15:57:00.260010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.259618 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 12 15:57:00.316074 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.315985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:00.445266 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.445231 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:00.595192 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.595111 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:57:00.595348 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.595324 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" containerID="cri-o://73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9" gracePeriod=30 Jul 12 15:57:00.606753 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.606712 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:00.808507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.808471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:00.961752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:00.961719 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:01.258959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:01.258870 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 12 15:57:01.258959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:01.258925 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 12 15:57:01.345035 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:01.345004 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:01.446346 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:01.446317 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:01.558022 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:01.557912 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:02.259321 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:02.259283 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 12 15:57:02.259535 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:02.259358 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 12 15:57:03.259184 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:03.259142 2563 patch_prober.go:28] interesting pod/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 12 15:57:03.259618 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:03.259205 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 12 15:57:04.262478 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:04.262446 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:57:04.262933 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:04.262723 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:57:04.263503 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:04.263487 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:57:04.691275 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:04.691237 2563 patch_prober.go:28] interesting pod/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 15:57:04.691456 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:04.691291 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 15:57:05.847576 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.847543 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:57:05.918872 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.918842 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerID="73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9" exitCode=0 Jul 12 15:57:05.919048 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.918888 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" Jul 12 15:57:05.919048 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.918924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerDied","Data":"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9"} Jul 12 15:57:05.919048 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.919009 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg" event={"ID":"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0","Type":"ContainerDied","Data":"3287b9e334de8c27c9c66e15f98a1dd456f2a2062cf0bcd6c80ad7774e5e50c7"} Jul 12 15:57:05.919048 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.919039 2563 scope.go:117] "RemoveContainer" containerID="73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9" Jul 12 15:57:05.927526 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.927504 2563 scope.go:117] "RemoveContainer" containerID="556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93" Jul 12 15:57:05.935747 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.935725 2563 scope.go:117] "RemoveContainer" containerID="73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9" Jul 12 15:57:05.936024 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:57:05.936006 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9\": container with ID starting with 73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9 not found: ID does not exist" containerID="73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9" Jul 12 15:57:05.936076 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.936035 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9"} err="failed to get container status \"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9\": rpc error: code = NotFound desc = could not find container \"73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9\": container with ID starting with 73e53a764a675184e9da6a78eb0e476bb94335a9dce64a701b8f7ff8909ec2b9 not found: ID does not exist" Jul 12 15:57:05.936076 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.936070 2563 scope.go:117] "RemoveContainer" containerID="556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93" Jul 12 15:57:05.936332 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:57:05.936307 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93\": container with ID starting with 556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93 not found: ID does not exist" containerID="556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93" Jul 12 15:57:05.936489 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.936336 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93"} err="failed to get container status \"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93\": rpc error: code = NotFound desc = could not find container \"556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93\": container with ID starting with 556a7d0461449aefcdf44f9737c6b2dc1699cff97c3e9aa4c1733f9662fa6f93 not found: ID does not exist" Jul 12 15:57:05.997550 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997507 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997561 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997578 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997614 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997629 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997657 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvsnx\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.997742 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997691 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.998061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997794 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.998061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.997824 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket\") pod \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\" (UID: \"5f16af0b-d1b5-4611-8f5c-c492bd93f9e0\") " Jul 12 15:57:05.998176 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.998117 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:57:05.998271 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.998247 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:05.998668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.998587 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:05.998668 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.998612 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:05.998970 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:05.998940 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:06.000128 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.000079 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:06.000236 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.000167 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:57:06.000236 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.000178 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:57:06.000236 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.000195 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx" (OuterVolumeSpecName: "kube-api-access-bvsnx") pod "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" (UID: "5f16af0b-d1b5-4611-8f5c-c492bd93f9e0"). InnerVolumeSpecName "kube-api-access-bvsnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:57:06.098764 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098731 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.098764 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098762 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.098764 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098771 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098780 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098791 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098804 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvsnx\" (UniqueName: \"kubernetes.io/projected/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-kube-api-access-bvsnx\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098813 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098823 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.099010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.098830 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:06.242038 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.242000 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:57:06.248008 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.247978 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a5af97-openshift-default-7776c67fb5-kjggg"] Jul 12 15:57:06.873800 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.873764 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:06.929093 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.929060 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" path="/var/lib/kubelet/pods/5f16af0b-d1b5-4611-8f5c-c492bd93f9e0/volumes" Jul 12 15:57:06.932859 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:06.932835 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:07.096563 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.096531 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:57:07.096770 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.096744 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" containerID="cri-o://cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351" gracePeriod=30 Jul 12 15:57:07.124923 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.124826 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:07.328113 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.328078 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:07.655039 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.655006 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:07.927607 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.927529 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 15:57:07.927959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.927806 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:07.927959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.927817 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:07.927959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.927881 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:07.927959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.927889 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:07.932449 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.932427 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:07.935050 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.935027 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a99179-openshift-default-dockercfg-zd7zg\"" Jul 12 15:57:07.942069 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.942044 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 15:57:07.972977 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:07.972940 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:08.116905 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.116866 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.116905 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.116900 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117132 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.116931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117132 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117011 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117132 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117132 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.117292 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.117177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69nfm\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.146984 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.146944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:08.218330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218229 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218330 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69nfm\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218446 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218744 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218793 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218900 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.218965 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.218941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.219162 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.219087 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.220895 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.220872 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.221118 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.221100 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.226620 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.226596 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.226620 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.226610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69nfm\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm\") pod \"e2e-shared-a-a99179-openshift-default-866f945d75-j94cf\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.243323 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.243286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:08.394328 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.394295 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 15:57:08.397660 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:57:08.397614 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a15bcc_fa83_4c0a_a624_103586b0055b.slice/crio-304711b9f448f565823ed7b54bb41a24ca91e407bad1f66132570292a775fd78 WatchSource:0}: Error finding container 304711b9f448f565823ed7b54bb41a24ca91e407bad1f66132570292a775fd78: Status 404 returned error can't find the container with id 304711b9f448f565823ed7b54bb41a24ca91e407bad1f66132570292a775fd78 Jul 12 15:57:08.399955 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.399926 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:08.400064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.399987 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:08.400064 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.400012 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:08.935637 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.935599 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerStarted","Data":"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e"} Jul 12 15:57:08.936123 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.935658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerStarted","Data":"304711b9f448f565823ed7b54bb41a24ca91e407bad1f66132570292a775fd78"} Jul 12 15:57:08.956080 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:08.956024 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podStartSLOduration=1.9560090350000001 podStartE2EDuration="1.956009035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:57:08.954244694 +0000 UTC m=+2214.504445480" watchObservedRunningTime="2026-07-12 15:57:08.956009035 +0000 UTC m=+2214.506209869" Jul 12 15:57:09.244124 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:09.244029 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:09.249050 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:09.249023 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:09.940748 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:09.940719 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:09.941645 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:09.941626 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:10.881263 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:10.881230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:10.964950 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:10.964919 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:11.030256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:11.030214 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:11.048276 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:11.048246 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:57:11.129982 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:11.129946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:11.413256 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:11.413230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:11.514079 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:11.514044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:12.355726 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.355666 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:57:12.458560 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458526 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458560 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458566 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458599 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458630 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458660 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458694 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458712 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ldcl\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458743 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.458817 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458758 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert\") pod \"6f42c849-5e09-4b49-86dc-bd64a71a3214\" (UID: \"6f42c849-5e09-4b49-86dc-bd64a71a3214\") " Jul 12 15:57:12.459196 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458886 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:12.459196 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.458918 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:12.459196 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.459031 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.459196 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.459052 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.459439 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.459316 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:12.459439 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.459342 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:57:12.459962 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.459942 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:12.461155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.461125 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 15:57:12.461155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.461136 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl" (OuterVolumeSpecName: "kube-api-access-9ldcl") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "kube-api-access-9ldcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:57:12.461626 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.461605 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:57:12.461626 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.461615 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f42c849-5e09-4b49-86dc-bd64a71a3214" (UID: "6f42c849-5e09-4b49-86dc-bd64a71a3214"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:57:12.559825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559789 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.559825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559820 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ldcl\" (UniqueName: \"kubernetes.io/projected/6f42c849-5e09-4b49-86dc-bd64a71a3214-kube-api-access-9ldcl\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.559825 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559830 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.560061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559839 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f42c849-5e09-4b49-86dc-bd64a71a3214-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.560061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559849 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.560061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559856 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.560061 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.559865 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f42c849-5e09-4b49-86dc-bd64a71a3214-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 15:57:12.956674 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.956637 2563 generic.go:356] "Generic (PLEG): container finished" podID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerID="cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351" exitCode=0 Jul 12 15:57:12.956851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.956693 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" Jul 12 15:57:12.956851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.956729 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerDied","Data":"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351"} Jul 12 15:57:12.956851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.956790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr" event={"ID":"6f42c849-5e09-4b49-86dc-bd64a71a3214","Type":"ContainerDied","Data":"f36d1eef9021be2b0d6e926fcdbd1b7d6bd1c5c0f18ea2af74dc9a4c70f6a78b"} Jul 12 15:57:12.956851 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.956814 2563 scope.go:117] "RemoveContainer" containerID="cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351" Jul 12 15:57:12.965787 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.965767 2563 scope.go:117] "RemoveContainer" containerID="21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1" Jul 12 15:57:12.974317 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.974296 2563 scope.go:117] "RemoveContainer" containerID="cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351" Jul 12 15:57:12.974646 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:57:12.974623 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351\": container with ID starting with cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351 not found: ID does not exist" containerID="cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351" Jul 12 15:57:12.974738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.974656 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351"} err="failed to get container status \"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351\": rpc error: code = NotFound desc = could not find container \"cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351\": container with ID starting with cc0f2ef4d799d727308ffec28eebf960741cf5b89c808a15b95f13b258fb7351 not found: ID does not exist" Jul 12 15:57:12.974738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.974684 2563 scope.go:117] "RemoveContainer" containerID="21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1" Jul 12 15:57:12.974738 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.974629 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:57:12.974973 ip-10-0-142-230 kubenswrapper[2563]: E0712 15:57:12.974955 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1\": container with ID starting with 21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1 not found: ID does not exist" containerID="21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1" Jul 12 15:57:12.975009 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.974979 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1"} err="failed to get container status \"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1\": rpc error: code = NotFound desc = could not find container \"21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1\": container with ID starting with 21dd43a632ead7724f4d9e695a65bc909a02afa68ce7f5d4b45ddc46d45969d1 not found: ID does not exist" Jul 12 15:57:12.978215 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:12.978187 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e6ad77-openshift-default-67f7b589f7-ngpxr"] Jul 12 15:57:14.929370 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:14.929339 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" path="/var/lib/kubelet/pods/6f42c849-5e09-4b49-86dc-bd64a71a3214/volumes" Jul 12 15:57:15.344892 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.344806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:15.665576 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.665547 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:15.940789 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.940692 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.940986 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.940997 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f16af0b-d1b5-4611-8f5c-c492bd93f9e0" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941016 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941021 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941030 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941036 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941080 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.941154 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.941087 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f42c849-5e09-4b49-86dc-bd64a71a3214" containerName="istio-proxy" Jul 12 15:57:15.945575 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.945556 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.949051 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.949031 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3da434-openshift-default-dockercfg-m2rkk\"" Jul 12 15:57:15.978534 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.978499 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 15:57:15.985079 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985047 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985092 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985252 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985168 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985483 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985483 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:15.985483 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:15.985324 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkm9w\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.010217 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.010177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:16.086011 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.085969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086011 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086039 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkm9w\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086286 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086879 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086879 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086585 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086879 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.086879 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.086734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.087067 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.087048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.088659 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.088637 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.088879 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.088859 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.097155 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.097132 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.097729 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.097704 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkm9w\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w\") pod \"e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.255581 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.255488 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:16.390422 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.390380 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 15:57:16.392693 ip-10-0-142-230 kubenswrapper[2563]: W0712 15:57:16.392665 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d1a150_c64a_443c_b260_8f7cb10fa91d.slice/crio-feb125f76eacb1eef8bf1549d3be9975952e4e2360cb1a5d3aabad9c86293f03 WatchSource:0}: Error finding container feb125f76eacb1eef8bf1549d3be9975952e4e2360cb1a5d3aabad9c86293f03: Status 404 returned error can't find the container with id feb125f76eacb1eef8bf1549d3be9975952e4e2360cb1a5d3aabad9c86293f03 Jul 12 15:57:16.394418 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.394374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:16.394758 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.394733 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:16.394812 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.394794 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:16.394847 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.394823 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:16.978507 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.978464 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerStarted","Data":"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5"} Jul 12 15:57:16.978985 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:16.978529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerStarted","Data":"feb125f76eacb1eef8bf1549d3be9975952e4e2360cb1a5d3aabad9c86293f03"} Jul 12 15:57:17.002201 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:17.002124 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podStartSLOduration=2.002102933 podStartE2EDuration="2.002102933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:57:17.000071028 +0000 UTC m=+2222.550271828" watchObservedRunningTime="2026-07-12 15:57:17.002102933 +0000 UTC m=+2222.552303711" Jul 12 15:57:17.256448 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:17.256320 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:17.261463 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:17.261432 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:17.983469 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:17.983439 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:17.984391 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:17.984373 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:18.951010 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:18.950971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:19.051654 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:19.051618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:19.226941 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:19.226803 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:19.445689 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:19.445654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:19.775572 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:19.775538 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:19.875585 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:19.875549 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:20.634440 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:20.634378 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:57:23.436613 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:23.436581 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:24.942264 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:24.942232 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 12 15:57:24.942752 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:24.942299 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 12 15:57:25.016006 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.015976 2563 generic.go:356] "Generic (PLEG): container finished" podID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerID="8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e" exitCode=0 Jul 12 15:57:25.016178 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.016055 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerDied","Data":"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e"} Jul 12 15:57:25.016494 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.016477 2563 scope.go:117] "RemoveContainer" containerID="8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e" Jul 12 15:57:25.017568 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.017534 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:25.017701 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.017624 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:25.017701 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:25.017671 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:26.028966 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:26.028928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerStarted","Data":"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c"} Jul 12 15:57:26.244324 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:26.244289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:26.244761 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:26.244676 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 12 15:57:26.244937 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:26.244915 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 12 15:57:27.244319 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:27.244278 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 12 15:57:27.244931 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:27.244895 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 12 15:57:28.244057 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:28.244014 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 12 15:57:28.244379 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:28.244346 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 12 15:57:29.038684 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:29.038632 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:57:29.244597 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:29.244550 2563 patch_prober.go:28] interesting pod/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 12 15:57:29.245112 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:29.244656 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 12 15:57:29.892455 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:29.892387 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:29.991000 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:29.990966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:30.248109 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:30.248082 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:30.248542 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:30.248426 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:30.249268 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:30.249245 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 15:57:32.154451 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:32.154414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:32.255515 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:32.255479 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:32.984713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:32.984629 2563 patch_prober.go:28] interesting pod/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 12 15:57:32.984713 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:32.984684 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 12 15:57:33.068039 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.068006 2563 generic.go:356] "Generic (PLEG): container finished" podID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerID="58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5" exitCode=0 Jul 12 15:57:33.068039 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.068043 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerDied","Data":"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5"} Jul 12 15:57:33.068481 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.068463 2563 scope.go:117] "RemoveContainer" containerID="58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5" Jul 12 15:57:33.069511 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.069480 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:33.069579 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.069566 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:33.069624 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:33.069612 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 12 15:57:34.073959 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:34.073922 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerStarted","Data":"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386"} Jul 12 15:57:34.256281 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:34.256238 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:34.256550 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:34.256527 2563 patch_prober.go:28] interesting pod/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 12 15:57:34.256609 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:34.256578 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 12 15:57:35.256278 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:35.256236 2563 patch_prober.go:28] interesting pod/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 12 15:57:35.256707 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:35.256295 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 12 15:57:36.256357 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:36.256314 2563 patch_prober.go:28] interesting pod/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 12 15:57:36.256784 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:36.256389 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 12 15:57:37.256037 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:37.256007 2563 patch_prober.go:28] interesting pod/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 12 15:57:37.256218 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:37.256060 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 12 15:57:38.260520 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:38.260482 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:38.260927 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:38.260731 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:38.261354 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:38.261336 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 15:57:40.404086 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:40.404053 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:40.503794 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:40.503761 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:44.819543 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:44.819500 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:57:47.933455 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:47.933417 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:57:57.749990 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:57:57.749950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:58:03.836060 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:03.836027 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:58:03.936210 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:03.936157 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:58:12.524263 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:12.524228 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:58:25.219736 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:25.219706 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:58:28.935550 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:28.935514 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:58:41.737615 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:41.737572 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:58:58.939374 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:58:58.939334 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:59:07.619224 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:07.619109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:59:09.011029 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:09.010988 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 15:59:12.537641 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:12.537605 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:59:25.649728 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:25.649694 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:59:40.438747 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:40.438709 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 15:59:56.248622 ip-10-0-142-230 kubenswrapper[2563]: I0712 15:59:56.248579 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:00:00.134743 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.134705 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8"] Jul 12 16:00:00.138147 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.138124 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:00.140967 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.140950 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 16:00:00.148375 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.148346 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8"] Jul 12 16:00:00.260796 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.260753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxlx\" (UniqueName: \"kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx\") pod \"maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8\" (UID: \"d51b47e2-1b78-437f-b594-f55dedb2ecc8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:00.361846 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.361804 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxlx\" (UniqueName: \"kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx\") pod \"maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8\" (UID: \"d51b47e2-1b78-437f-b594-f55dedb2ecc8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:00.369824 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.369786 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxlx\" (UniqueName: \"kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx\") pod \"maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8\" (UID: \"d51b47e2-1b78-437f-b594-f55dedb2ecc8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:00.449891 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.449857 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:00.581079 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.581054 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8"] Jul 12 16:00:00.583572 ip-10-0-142-230 kubenswrapper[2563]: W0712 16:00:00.583544 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd51b47e2_1b78_437f_b594_f55dedb2ecc8.slice/crio-80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b WatchSource:0}: Error finding container 80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b: Status 404 returned error can't find the container with id 80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b Jul 12 16:00:00.702566 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:00.702480 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" event={"ID":"d51b47e2-1b78-437f-b594-f55dedb2ecc8","Type":"ContainerStarted","Data":"80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b"} Jul 12 16:00:01.709772 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:01.709736 2563 generic.go:356] "Generic (PLEG): container finished" podID="d51b47e2-1b78-437f-b594-f55dedb2ecc8" containerID="9e617a710477289be832b678bdc0ebff18d7a5d434e45e83fd518d6668878403" exitCode=0 Jul 12 16:00:01.710185 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:01.709825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" event={"ID":"d51b47e2-1b78-437f-b594-f55dedb2ecc8","Type":"ContainerDied","Data":"9e617a710477289be832b678bdc0ebff18d7a5d434e45e83fd518d6668878403"} Jul 12 16:00:02.837609 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:02.837580 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:02.988543 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:02.988455 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgxlx\" (UniqueName: \"kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx\") pod \"d51b47e2-1b78-437f-b594-f55dedb2ecc8\" (UID: \"d51b47e2-1b78-437f-b594-f55dedb2ecc8\") " Jul 12 16:00:02.990618 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:02.990595 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx" (OuterVolumeSpecName: "kube-api-access-tgxlx") pod "d51b47e2-1b78-437f-b594-f55dedb2ecc8" (UID: "d51b47e2-1b78-437f-b594-f55dedb2ecc8"). InnerVolumeSpecName "kube-api-access-tgxlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:00:03.089825 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:03.089788 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgxlx\" (UniqueName: \"kubernetes.io/projected/d51b47e2-1b78-437f-b594-f55dedb2ecc8-kube-api-access-tgxlx\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:00:03.719564 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:03.719531 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" Jul 12 16:00:03.719779 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:03.719534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8" event={"ID":"d51b47e2-1b78-437f-b594-f55dedb2ecc8","Type":"ContainerDied","Data":"80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b"} Jul 12 16:00:03.719779 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:03.719651 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80171807ebd7edb4a911283c4717109563c536712ec538124d4634fa74e2f67b" Jul 12 16:00:11.648153 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:11.648109 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:00:14.968871 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:14.968841 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 16:00:14.978855 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:14.978830 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 16:00:16.608793 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:16.608755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:00:17.418073 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:17.418044 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:00:28.349288 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:28.349248 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:00:29.623029 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:29.622988 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:00:39.036905 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:39.036824 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:00:57.231733 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:00:57.231700 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:01:01.038340 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:01.038308 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:01:15.049379 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:15.049344 2563 scope.go:117] "RemoveContainer" containerID="ab13bcd71c4e53ef373b5e3b28bcd81a42ac353e1e4a432ac2cee9bd89aaab5f" Jul 12 16:01:20.135088 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:20.135050 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:01:34.851137 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:34.851101 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:01:39.838593 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:39.838560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:01:40.698848 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:40.698818 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:01:51.536442 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:51.536389 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:01:52.508512 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:01:52.508474 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:02.959855 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:02.959821 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:02:13.508809 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:13.508768 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:18.234365 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:18.234332 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:02:20.678643 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:20.678611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:30.543684 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:30.543650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:30.646323 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:30.646270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:32.755007 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:32.754853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:32.843601 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:32.843564 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:02:32.848619 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:32.848583 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:41.137383 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:41.137340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:41.238486 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:41.238424 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:50.344566 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:50.344527 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:02:54.326051 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:54.326016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:55.197709 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:55.197672 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:56.177569 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.177535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:56.543731 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.543642 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:56.643927 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.643897 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:56.758893 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.758857 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8"] Jul 12 16:02:56.763698 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.763673 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-a99179-29731200-6hrk8"] Jul 12 16:02:56.764545 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.764523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:02:56.928832 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:02:56.928802 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d51b47e2-1b78-437f-b594-f55dedb2ecc8" path="/var/lib/kubelet/pods/d51b47e2-1b78-437f-b594-f55dedb2ecc8/volumes" Jul 12 16:03:01.640106 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:01.640068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:05.942137 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:05.942096 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:03:38.376610 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:38.376531 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:38.625797 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:38.625758 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 16:03:38.625995 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:38.625972 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" containerID="cri-o://0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c" gracePeriod=30 Jul 12 16:03:38.645264 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:38.645186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:38.872432 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:38.872386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:39.039526 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:39.039487 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:39.433610 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:39.433574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:39.530793 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:39.530761 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:39.646096 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:39.646059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:42.736771 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:42.736733 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wg68v"] Jul 12 16:03:43.875529 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.875500 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 16:03:43.937189 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937137 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937198 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937224 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937261 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937330 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937353 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69nfm\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937418 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937392 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937438 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo\") pod \"c6a15bcc-fa83-4c0a-a624-103586b0055b\" (UID: \"c6a15bcc-fa83-4c0a-a624-103586b0055b\") " Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937590 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937619 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937699 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937628 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:43.937792 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.937694 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:43.938529 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.938497 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:43.939837 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.939800 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 16:03:43.939837 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.939825 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:43.940012 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.939888 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:43.940012 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:43.939933 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm" (OuterVolumeSpecName: "kube-api-access-69nfm") pod "c6a15bcc-fa83-4c0a-a624-103586b0055b" (UID: "c6a15bcc-fa83-4c0a-a624-103586b0055b"). InnerVolumeSpecName "kube-api-access-69nfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:44.038851 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038808 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.038851 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038841 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.038851 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038851 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.038851 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038860 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.039130 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038868 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69nfm\" (UniqueName: \"kubernetes.io/projected/c6a15bcc-fa83-4c0a-a624-103586b0055b-kube-api-access-69nfm\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.039130 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038878 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.039130 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038886 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6a15bcc-fa83-4c0a-a624-103586b0055b-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.039130 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.038894 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6a15bcc-fa83-4c0a-a624-103586b0055b-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:44.700302 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.700268 2563 generic.go:356] "Generic (PLEG): container finished" podID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerID="0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c" exitCode=0 Jul 12 16:03:44.700499 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.700319 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" Jul 12 16:03:44.700499 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.700326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerDied","Data":"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c"} Jul 12 16:03:44.700499 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.700369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf" event={"ID":"c6a15bcc-fa83-4c0a-a624-103586b0055b","Type":"ContainerDied","Data":"304711b9f448f565823ed7b54bb41a24ca91e407bad1f66132570292a775fd78"} Jul 12 16:03:44.700499 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.700391 2563 scope.go:117] "RemoveContainer" containerID="0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c" Jul 12 16:03:44.709204 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.709185 2563 scope.go:117] "RemoveContainer" containerID="8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e" Jul 12 16:03:44.717070 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.717050 2563 scope.go:117] "RemoveContainer" containerID="0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c" Jul 12 16:03:44.717347 ip-10-0-142-230 kubenswrapper[2563]: E0712 16:03:44.717325 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c\": container with ID starting with 0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c not found: ID does not exist" containerID="0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c" Jul 12 16:03:44.717414 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.717356 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c"} err="failed to get container status \"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c\": rpc error: code = NotFound desc = could not find container \"0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c\": container with ID starting with 0552738b7aacd5279953adee31e49a19d5d13cfa1c093462bc29a33a06eb5c4c not found: ID does not exist" Jul 12 16:03:44.717414 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.717376 2563 scope.go:117] "RemoveContainer" containerID="8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e" Jul 12 16:03:44.717652 ip-10-0-142-230 kubenswrapper[2563]: E0712 16:03:44.717634 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e\": container with ID starting with 8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e not found: ID does not exist" containerID="8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e" Jul 12 16:03:44.717696 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.717658 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e"} err="failed to get container status \"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e\": rpc error: code = NotFound desc = could not find container \"8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e\": container with ID starting with 8f3d6537fd1f200dd5093f54883bf6be0f7527bf0e1c7b4cfc23b3bcece3304e not found: ID does not exist" Jul 12 16:03:44.721557 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.721537 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 16:03:44.724876 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.724855 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a99179-openshift-default-866f945d75-j94cf"] Jul 12 16:03:44.929133 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:44.929097 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" path="/var/lib/kubelet/pods/c6a15bcc-fa83-4c0a-a624-103586b0055b/volumes" Jul 12 16:03:45.196104 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.196073 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:45.348917 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.348885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:45.463355 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.463275 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 16:03:45.463545 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.463517 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" containerID="cri-o://b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386" gracePeriod=30 Jul 12 16:03:45.487314 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.487278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:45.746154 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:45.746055 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:03:50.710848 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.710819 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 16:03:50.730078 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.729991 2563 generic.go:356] "Generic (PLEG): container finished" podID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerID="b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386" exitCode=0 Jul 12 16:03:50.730078 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.730042 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" Jul 12 16:03:50.730253 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.730070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerDied","Data":"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386"} Jul 12 16:03:50.730253 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.730135 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj" event={"ID":"54d1a150-c64a-443c-b260-8f7cb10fa91d","Type":"ContainerDied","Data":"feb125f76eacb1eef8bf1549d3be9975952e4e2360cb1a5d3aabad9c86293f03"} Jul 12 16:03:50.730253 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.730168 2563 scope.go:117] "RemoveContainer" containerID="b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386" Jul 12 16:03:50.739958 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.739936 2563 scope.go:117] "RemoveContainer" containerID="58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5" Jul 12 16:03:50.749740 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.749717 2563 scope.go:117] "RemoveContainer" containerID="b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386" Jul 12 16:03:50.750036 ip-10-0-142-230 kubenswrapper[2563]: E0712 16:03:50.750015 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386\": container with ID starting with b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386 not found: ID does not exist" containerID="b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386" Jul 12 16:03:50.750114 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.750048 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386"} err="failed to get container status \"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386\": rpc error: code = NotFound desc = could not find container \"b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386\": container with ID starting with b2983ec7ae77010e2a44061f3a5ff06359d346ea184db43e220eb9f08ee97386 not found: ID does not exist" Jul 12 16:03:50.750114 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.750075 2563 scope.go:117] "RemoveContainer" containerID="58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5" Jul 12 16:03:50.750362 ip-10-0-142-230 kubenswrapper[2563]: E0712 16:03:50.750343 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5\": container with ID starting with 58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5 not found: ID does not exist" containerID="58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5" Jul 12 16:03:50.750422 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.750368 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5"} err="failed to get container status \"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5\": rpc error: code = NotFound desc = could not find container \"58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5\": container with ID starting with 58f3596282851433d29a7fef806c559bef8176aa56ed12bbbc31384e52e323d5 not found: ID does not exist" Jul 12 16:03:50.793588 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793557 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.793752 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793603 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.793752 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793640 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.793752 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793667 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkm9w\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.793752 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793690 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.793752 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793720 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.794000 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793753 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.794000 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793777 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.794000 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793804 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo\") pod \"54d1a150-c64a-443c-b260-8f7cb10fa91d\" (UID: \"54d1a150-c64a-443c-b260-8f7cb10fa91d\") " Jul 12 16:03:50.794000 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.793934 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:50.794202 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.794067 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:50.794202 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.794184 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:50.794292 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.794254 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 16:03:50.794826 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.794796 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data" (OuterVolumeSpecName: "istio-data") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:50.796093 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.796068 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token" (OuterVolumeSpecName: "istio-token") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:50.796327 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.796301 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 16:03:50.796504 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.796485 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:50.796504 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.796492 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w" (OuterVolumeSpecName: "kube-api-access-pkm9w") pod "54d1a150-c64a-443c-b260-8f7cb10fa91d" (UID: "54d1a150-c64a-443c-b260-8f7cb10fa91d"). InnerVolumeSpecName "kube-api-access-pkm9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:50.894739 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894694 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-data\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.894739 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894730 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-credential-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.894739 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894746 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-podinfo\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894758 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-certs\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894768 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-token\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894779 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-workload-socket\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894790 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkm9w\" (UniqueName: \"kubernetes.io/projected/54d1a150-c64a-443c-b260-8f7cb10fa91d-kube-api-access-pkm9w\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894803 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d1a150-c64a-443c-b260-8f7cb10fa91d-istiod-ca-cert\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.895009 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.894814 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d1a150-c64a-443c-b260-8f7cb10fa91d-istio-envoy\") on node \"ip-10-0-142-230.ec2.internal\" DevicePath \"\"" Jul 12 16:03:50.916522 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:50.916478 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29731185-l5ssh_a5012b28-5c9d-4949-a0c1-6b77763e84e2/cleanup/0.log" Jul 12 16:03:51.046199 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:51.046118 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 16:03:51.052125 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:51.052090 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3da434-openshift-default-7dbff76595-fwntj"] Jul 12 16:03:51.702544 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:51.702512 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-lw62v_422bc9f3-bcd3-4270-bb2d-41b96b2e6beb/copy-manifests/0.log" Jul 12 16:03:51.714170 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:51.714138 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-lw62v_422bc9f3-bcd3-4270-bb2d-41b96b2e6beb/manager/0.log" Jul 12 16:03:52.929007 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:52.928970 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" path="/var/lib/kubelet/pods/54d1a150-c64a-443c-b260-8f7cb10fa91d/volumes" Jul 12 16:03:53.751976 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:53.751948 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wg68v_c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6/limitador/0.log" Jul 12 16:03:54.307369 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:54.307339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4wq5k_095647c1-a4c1-4b34-88f5-e58a51e4bb2f/discovery/0.log" Jul 12 16:03:55.622486 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:55.622449 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2drcz_0ef87217-8dea-46b9-b2f6-b9d3d290ea00/main/0.log" Jul 12 16:03:59.917185 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:03:59.917142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:04:00.017310 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:00.017278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" limit=1024 Jul 12 16:04:02.844003 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:02.843966 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h7cqj_a1bf5087-3ddd-4030-bdca-c5299683ad1d/global-pull-secret-syncer/0.log" Jul 12 16:04:02.896321 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:02.896289 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-42264_d3c7a6a4-f419-42e8-be7a-f64e2d254290/konnectivity-agent/0.log" Jul 12 16:04:03.029295 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:03.029260 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-230.ec2.internal_a5a3ddb42b509e449091e6f0bf0e70c5/haproxy/0.log" Jul 12 16:04:08.819924 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:08.819891 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wg68v_c166bdb8-fe5d-44e0-8c89-74a6dba3a3e6/limitador/0.log" Jul 12 16:04:10.930815 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:10.930781 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66ssc_197f34d3-2a2e-4ab1-8f9a-e5665bef9360/node-exporter/0.log" Jul 12 16:04:10.950701 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:10.950671 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66ssc_197f34d3-2a2e-4ab1-8f9a-e5665bef9360/kube-rbac-proxy/0.log" Jul 12 16:04:10.969425 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:10.969383 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66ssc_197f34d3-2a2e-4ab1-8f9a-e5665bef9360/init-textfile/0.log" Jul 12 16:04:13.000135 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.000105 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-2gmjr_4ea66087-0992-41de-87d9-59515e7eb1d6/networking-console-plugin/0.log" Jul 12 16:04:13.524054 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524019 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw"] Jul 12 16:04:13.524374 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524357 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524376 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524386 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51b47e2-1b78-437f-b594-f55dedb2ecc8" containerName="cleanup" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524391 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51b47e2-1b78-437f-b594-f55dedb2ecc8" containerName="cleanup" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524421 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524430 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524442 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524448 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" Jul 12 16:04:13.524467 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524467 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.524746 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524473 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.524746 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524517 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d1a150-c64a-443c-b260-8f7cb10fa91d" containerName="istio-proxy" Jul 12 16:04:13.524746 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524525 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.524746 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524531 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51b47e2-1b78-437f-b594-f55dedb2ecc8" containerName="cleanup" Jul 12 16:04:13.524746 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.524538 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6a15bcc-fa83-4c0a-a624-103586b0055b" containerName="istio-proxy" Jul 12 16:04:13.527749 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.527724 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.530440 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.530418 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nth9q\"/\"openshift-service-ca.crt\"" Jul 12 16:04:13.530440 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.530392 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nth9q\"/\"kube-root-ca.crt\"" Jul 12 16:04:13.531536 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.531517 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nth9q\"/\"default-dockercfg-p7gsx\"" Jul 12 16:04:13.537251 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.537224 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw"] Jul 12 16:04:13.669825 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.669789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-lib-modules\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.670007 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.669850 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-podres\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.670007 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.669958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-proc\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.670007 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.670001 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d7l2\" (UniqueName: \"kubernetes.io/projected/8c213898-be1c-4dc2-803a-9a83ab60dc4d-kube-api-access-6d7l2\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.670168 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.670025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-sys\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771020 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.770983 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-proc\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d7l2\" (UniqueName: \"kubernetes.io/projected/8c213898-be1c-4dc2-803a-9a83ab60dc4d-kube-api-access-6d7l2\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771051 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-sys\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-lib-modules\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771108 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-proc\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-podres\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771186 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771174 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-sys\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771451 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771217 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-lib-modules\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.771451 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.771261 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8c213898-be1c-4dc2-803a-9a83ab60dc4d-podres\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.778810 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.778746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d7l2\" (UniqueName: \"kubernetes.io/projected/8c213898-be1c-4dc2-803a-9a83ab60dc4d-kube-api-access-6d7l2\") pod \"perf-node-gather-daemonset-48smw\" (UID: \"8c213898-be1c-4dc2-803a-9a83ab60dc4d\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.838557 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.838517 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:13.969301 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:13.969243 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw"] Jul 12 16:04:13.975460 ip-10-0-142-230 kubenswrapper[2563]: W0712 16:04:13.972609 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8c213898_be1c_4dc2_803a_9a83ab60dc4d.slice/crio-5dc691cc090f9d677dc85bddbd9fb22b3ee8c967495bc883527bdc11ff666340 WatchSource:0}: Error finding container 5dc691cc090f9d677dc85bddbd9fb22b3ee8c967495bc883527bdc11ff666340: Status 404 returned error can't find the container with id 5dc691cc090f9d677dc85bddbd9fb22b3ee8c967495bc883527bdc11ff666340 Jul 12 16:04:14.833676 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:14.833641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" event={"ID":"8c213898-be1c-4dc2-803a-9a83ab60dc4d","Type":"ContainerStarted","Data":"802ef5ac51faab63aab07a7233346c5a55bd9fd547792ef8b5e7b2d22bc0ecd2"} Jul 12 16:04:14.834090 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:14.833695 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" event={"ID":"8c213898-be1c-4dc2-803a-9a83ab60dc4d","Type":"ContainerStarted","Data":"5dc691cc090f9d677dc85bddbd9fb22b3ee8c967495bc883527bdc11ff666340"} Jul 12 16:04:14.834090 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:14.833746 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:14.850465 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:14.850412 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" podStartSLOduration=1.850379383 podStartE2EDuration="1.850379383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 16:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 16:04:14.849326359 +0000 UTC m=+2640.399527153" watchObservedRunningTime="2026-07-12 16:04:14.850379383 +0000 UTC m=+2640.400580158" Jul 12 16:04:15.730046 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:15.730014 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ckjzp_f0733aa5-675c-41c8-b924-87c2b65fb004/dns/0.log" Jul 12 16:04:15.749017 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:15.748990 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ckjzp_f0733aa5-675c-41c8-b924-87c2b65fb004/kube-rbac-proxy/0.log" Jul 12 16:04:15.859542 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:15.859512 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-77n27_ca059fb5-eee1-4767-bb92-d39f488b76a2/dns-node-resolver/0.log" Jul 12 16:04:16.514980 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:16.514943 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b6qnh_502c5db6-ee40-4043-84d1-b582261cc920/node-ca/0.log" Jul 12 16:04:17.597365 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:17.597329 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4wq5k_095647c1-a4c1-4b34-88f5-e58a51e4bb2f/discovery/0.log" Jul 12 16:04:18.482919 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:18.482881 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xgtjd_9f686a4e-6106-4ce9-8438-fefae0571b84/serve-healthcheck-canary/0.log" Jul 12 16:04:18.942507 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:18.942478 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9r594_1142c0f9-4f1f-449a-a1c1-e109015a7e53/kube-rbac-proxy/0.log" Jul 12 16:04:18.962717 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:18.962692 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9r594_1142c0f9-4f1f-449a-a1c1-e109015a7e53/exporter/0.log" Jul 12 16:04:18.981159 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:18.981135 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9r594_1142c0f9-4f1f-449a-a1c1-e109015a7e53/extractor/0.log" Jul 12 16:04:20.847764 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:20.847735 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-48smw" Jul 12 16:04:21.358681 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:21.358653 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29731185-l5ssh_a5012b28-5c9d-4949-a0c1-6b77763e84e2/cleanup/0.log" Jul 12 16:04:21.724181 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:21.724148 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-lw62v_422bc9f3-bcd3-4270-bb2d-41b96b2e6beb/manager/0.log" Jul 12 16:04:21.739161 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:21.739130 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-lw62v_422bc9f3-bcd3-4270-bb2d-41b96b2e6beb/copy-manifests/0.log" Jul 12 16:04:23.270659 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:23.270584 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59c7cf6567-xtqvc_09e59495-b659-4129-a4cf-449c3a3c50dc/manager/0.log" Jul 12 16:04:24.119175 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:24.119094 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4wq5k_095647c1-a4c1-4b34-88f5-e58a51e4bb2f/discovery/0.log" Jul 12 16:04:25.550016 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:25.549982 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4wq5k_095647c1-a4c1-4b34-88f5-e58a51e4bb2f/discovery/0.log" Jul 12 16:04:29.474833 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.474800 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-75kzc_be136fa6-3d60-4905-9ffb-4a9528a0d7fc/kube-multus/0.log" Jul 12 16:04:29.496507 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.496475 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/kube-multus-additional-cni-plugins/0.log" Jul 12 16:04:29.517674 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.517646 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/egress-router-binary-copy/0.log" Jul 12 16:04:29.535631 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.535608 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/cni-plugins/0.log" Jul 12 16:04:29.555286 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.555264 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/bond-cni-plugin/0.log" Jul 12 16:04:29.573957 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.573931 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/routeoverride-cni/0.log" Jul 12 16:04:29.591620 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.591594 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/whereabouts-cni-bincopy/0.log" Jul 12 16:04:29.609247 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:29.609216 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gktwq_b2bc4f44-cbfe-4b48-9157-a80b31cbffea/whereabouts-cni/0.log" Jul 12 16:04:30.012116 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.012083 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dtq8d_733d8099-0f3f-4195-a5fd-4813d320ce42/network-metrics-daemon/0.log" Jul 12 16:04:30.029018 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.028991 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dtq8d_733d8099-0f3f-4195-a5fd-4813d320ce42/kube-rbac-proxy/0.log" Jul 12 16:04:30.836788 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.836741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-controller/0.log" Jul 12 16:04:30.853725 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.853694 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/0.log" Jul 12 16:04:30.853725 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.853708 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovn-acl-logging/1.log" Jul 12 16:04:30.871307 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.871284 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/kube-rbac-proxy-node/0.log" Jul 12 16:04:30.889858 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.889833 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 16:04:30.904971 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.904946 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/northd/0.log" Jul 12 16:04:30.923409 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.923380 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/nbdb/0.log" Jul 12 16:04:30.941454 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.941386 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/sbdb/0.log" Jul 12 16:04:30.989856 ip-10-0-142-230 kubenswrapper[2563]: I0712 16:04:30.989820 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-58qlj_8578cb3b-be11-4fd2-8811-e44eaa5f0536/ovnkube-controller/0.log"