Jul 03 14:06:20.939422 ip-10-0-132-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 03 14:06:20.939431 ip-10-0-132-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 03 14:06:20.939438 ip-10-0-132-213 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 03 14:06:20.939707 ip-10-0-132-213 systemd[1]: Failed to start Kubernetes Kubelet. Jul 03 14:06:31.008772 ip-10-0-132-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 03 14:06:31.008790 ip-10-0-132-213 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 309d4b38a3f14e2eaa813a6a8bcde1f5 -- Jul 03 14:08:26.463016 ip-10-0-132-213 systemd[1]: Starting Kubernetes Kubelet... Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.851022 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856886 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856898 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856904 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856908 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856912 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856915 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856918 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:26.885515 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856920 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856924 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856927 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856930 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856932 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856935 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856938 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856941 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856943 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856946 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856956 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856959 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856962 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856964 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856967 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856970 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856973 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856977 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856980 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856983 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:26.886386 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856986 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856989 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856992 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856994 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.856997 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857000 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857003 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857005 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857008 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857011 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857014 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857016 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857020 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857023 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857026 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857029 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857031 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857034 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857037 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857039 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:26.887686 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857042 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857044 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857047 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857050 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857052 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857055 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857058 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857060 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857064 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857067 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857069 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857071 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857074 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857077 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857080 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857083 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857085 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857087 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857090 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:26.888340 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857092 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857096 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857098 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857101 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857103 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857106 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857109 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857111 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857114 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857117 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857120 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857123 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857125 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857128 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857130 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857132 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857136 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857138 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857141 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857143 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:26.890073 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857146 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857149 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857151 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857154 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857156 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857159 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857162 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857165 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857167 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857169 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857172 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857174 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857178 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857873 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857880 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857884 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857886 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857890 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857892 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:26.890745 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857895 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857897 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857900 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857903 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857906 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857909 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857911 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857914 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857917 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857919 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857922 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857926 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857930 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857933 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857936 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857939 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857941 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857944 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857947 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:26.893110 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857949 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857952 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857955 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857957 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857960 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857963 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857966 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857969 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857971 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857974 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857976 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857979 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857981 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857984 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857987 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857989 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857992 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857995 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.857998 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858001 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:26.893753 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858004 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858007 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858010 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858013 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858016 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858019 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858022 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858024 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858027 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858030 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858032 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858035 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858037 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858040 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858042 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858044 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858047 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858050 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858052 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858055 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:26.895691 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858057 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858060 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858062 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858065 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858067 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858070 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858072 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858074 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858078 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858081 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858085 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858088 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858093 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858095 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858098 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858101 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858104 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858107 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858109 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858112 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:26.896339 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858114 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858117 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858119 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858122 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858124 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858127 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858130 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858132 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858135 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858137 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858140 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858143 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858146 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858149 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858502 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858507 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858510 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858513 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858516 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858518 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:26.898951 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858521 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858523 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858527 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858529 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858532 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858535 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858538 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858540 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858543 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858546 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858548 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858551 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858553 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858556 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858558 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858561 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858563 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858566 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858569 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858571 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:26.899610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858574 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858577 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858579 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858582 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858585 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858588 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858590 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858593 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858595 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858598 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858600 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858603 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858605 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858607 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858611 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858614 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858616 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858620 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858624 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:26.902396 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858627 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858630 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858633 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858635 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858638 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858641 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858643 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858645 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858649 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858655 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858658 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858660 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858663 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858665 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858668 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858671 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858674 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858677 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858679 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:26.903035 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858682 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858684 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858687 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858690 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858693 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858695 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858698 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858700 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858703 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858706 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858712 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858715 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858717 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858720 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858736 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858754 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858765 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858770 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858774 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:26.905158 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858777 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858780 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858782 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858786 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858789 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858792 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858794 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858797 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858800 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858802 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858805 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858807 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858810 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858812 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858815 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.858817 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858880 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858891 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858897 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858902 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858906 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 14:08:26.906010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858910 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858914 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858918 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858922 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858925 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858928 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858932 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858935 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858938 2580 flags.go:64] FLAG: --cgroup-root="" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858940 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858943 2580 flags.go:64] FLAG: --client-ca-file="" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858947 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858950 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858954 2580 flags.go:64] FLAG: --cluster-domain="" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858959 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858962 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858966 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858969 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858972 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858976 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858979 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858982 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858985 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858988 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 14:08:26.908219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858991 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858994 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.858997 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859001 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859004 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859007 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859010 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859013 2580 flags.go:64] FLAG: --enable-server="true" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859016 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859025 2580 flags.go:64] FLAG: --event-burst="100" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859028 2580 flags.go:64] FLAG: --event-qps="50" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859031 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859040 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859043 2580 flags.go:64] FLAG: --eviction-hard="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859046 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859049 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859052 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859055 2580 flags.go:64] FLAG: --eviction-soft="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859058 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859060 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859063 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859066 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859071 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859074 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859077 2580 flags.go:64] FLAG: --feature-gates="" Jul 03 14:08:26.909091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859080 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859083 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859086 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859089 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859092 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859095 2580 flags.go:64] FLAG: --help="false" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859097 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-132-213.ec2.internal" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859100 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859103 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859106 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859110 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859113 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859116 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859119 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859121 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859124 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859128 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859131 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859134 2580 flags.go:64] FLAG: --kube-reserved="" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859137 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859141 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859145 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859147 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859150 2580 flags.go:64] FLAG: --lock-file="" Jul 03 14:08:26.912106 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859153 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859156 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859159 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859164 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859167 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859171 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859174 2580 flags.go:64] FLAG: --logging-format="text" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859177 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859180 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859183 2580 flags.go:64] FLAG: --manifest-url="" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859186 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859190 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859193 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859197 2580 flags.go:64] FLAG: --max-pods="110" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859200 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859203 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859206 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859209 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859212 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859214 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859217 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859223 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859227 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859229 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 14:08:26.912982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859234 2580 flags.go:64] FLAG: --pod-cidr="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859237 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859242 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859245 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859248 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859251 2580 flags.go:64] FLAG: --port="10250" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859254 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859257 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b68278a732f6374d" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859260 2580 flags.go:64] FLAG: --qos-reserved="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859263 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859266 2580 flags.go:64] FLAG: --register-node="true" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859269 2580 flags.go:64] FLAG: --register-with-taints="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859274 2580 flags.go:64] FLAG: --registry-burst="10" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859278 2580 flags.go:64] FLAG: --registry-qps="5" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859281 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859284 2580 flags.go:64] FLAG: --reserved-memory="" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859287 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859290 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859293 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859296 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859299 2580 flags.go:64] FLAG: --runonce="false" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859302 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859305 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859308 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859311 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 14:08:26.984280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859314 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 14:08:26.976561 ip-10-0-132-213 systemd[1]: Started Kubernetes Kubelet. Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859317 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859319 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859322 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859325 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859328 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859330 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859333 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859338 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859341 2580 flags.go:64] FLAG: --system-cgroups="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859344 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859350 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859353 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859356 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859361 2580 flags.go:64] FLAG: --tls-min-version="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859364 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859366 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859369 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859372 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859375 2580 flags.go:64] FLAG: --v="2" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859380 2580 flags.go:64] FLAG: --version="false" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859384 2580 flags.go:64] FLAG: --vmodule="" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859388 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.859391 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859505 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:26.985311 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859509 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859512 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859515 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859517 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859520 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859523 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859525 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859528 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859531 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859534 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859536 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859539 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859541 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859544 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859546 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859551 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859554 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859558 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859561 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:26.987092 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859565 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859568 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859571 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859574 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859577 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859580 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859583 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859586 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859591 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859594 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859596 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859599 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859602 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859604 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859607 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859610 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859613 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859615 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859618 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859620 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:26.988171 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859623 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859625 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859628 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859630 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859633 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859636 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859639 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859641 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859645 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859647 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859651 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859655 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859658 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859660 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859663 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859669 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859672 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859674 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859677 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:26.989308 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859680 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859683 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859686 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859689 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859691 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859694 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859696 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859699 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859701 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859704 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859706 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859709 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859711 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859714 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859717 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859719 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859722 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859738 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859741 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859743 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:26.990081 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859746 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859750 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859752 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859755 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859758 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859761 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859763 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859766 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859768 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859772 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859774 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859777 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859780 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859783 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859786 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859789 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859792 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859794 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859797 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:26.992143 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.859800 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.860437 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.866999 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867014 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867026 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867031 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867086 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867091 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867094 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867098 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867102 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867106 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867109 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867112 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:26.993029 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867115 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867118 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867122 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867125 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867128 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867131 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867133 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867136 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867139 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867142 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867145 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867147 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867151 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867153 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867156 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867159 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867162 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867164 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867167 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867169 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:26.996056 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867172 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867175 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867178 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867180 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867183 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867186 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867188 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867191 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867193 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867196 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867199 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867201 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867204 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867207 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867210 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867212 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867215 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867217 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867221 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867223 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:26.997432 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867226 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867229 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867232 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867234 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867237 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867240 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867242 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867245 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867248 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867250 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867253 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867255 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867258 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867263 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867266 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867269 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867272 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867276 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867279 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:26.998683 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867282 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867286 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867290 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867293 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867296 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867298 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867301 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867303 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867306 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867309 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867311 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867314 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867317 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867319 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867323 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867326 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867329 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867332 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867334 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:27.002003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867337 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867341 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867343 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867346 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867348 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867351 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867353 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867357 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867360 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867362 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867365 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867368 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867370 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867375 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867483 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 14:08:27.002852 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867489 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867492 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867495 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867498 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867501 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867504 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867507 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867509 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867512 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867514 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867517 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867520 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867524 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867527 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867530 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867533 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867536 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867539 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867543 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 14:08:27.003512 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867545 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867548 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867550 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867553 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867555 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867558 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867561 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867564 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867567 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867569 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867572 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867574 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867577 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867580 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867583 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867585 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867588 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867590 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867593 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 14:08:27.004273 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867596 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867598 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867601 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867603 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867606 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867608 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867611 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867613 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867616 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867619 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867621 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867625 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867630 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867633 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867636 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867638 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867641 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867644 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867646 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867650 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 14:08:27.005047 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867653 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867655 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867658 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867661 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867663 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867666 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867669 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867671 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867674 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867676 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867679 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867682 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867684 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867687 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867690 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867692 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867695 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867698 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867700 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867703 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 14:08:27.005885 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867705 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867708 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867711 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867714 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867718 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867721 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867741 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867744 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867747 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867750 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867753 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867757 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867759 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867762 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867764 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867767 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867769 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867772 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867775 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 14:08:27.087165 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:26.867778 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867784 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.867887 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.870014 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.871208 2580 server.go:1019] "Starting client certificate rotation" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.873149 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.878407 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.878427 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.878472 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.926021 2580 log.go:25] "Validated CRI v1 runtime API" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.928643 2580 log.go:25] "Validated CRI v1 image API" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.944656 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 14:08:27.087817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.950718 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 14:08:27.090169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.952841 2580 fs.go:135] Filesystem UUIDs: map[28a8a6a8-93af-4f51-8035-a848c0dfd757:/dev/nvme0n1p4 78ff65b1-b791-4fc3-ae45-3ddec4ded968:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 03 14:08:27.090169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.952863 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.957481 2580 manager.go:216] Machine: {Timestamp:2026-07-03 14:08:26.956976506 +0000 UTC m=+0.378740976 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3148101 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a659db31c032be85d40165a3f92e0 SystemUUID:ec2a659d-b31c-032b-e85d-40165a3f92e0 BootID:309d4b38-a3f1-4e2e-aa81-3a6a8bcde1f5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:95:8d:a8:9e:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:95:8d:a8:9e:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:e6:8a:46:01:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.957576 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.957650 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.960158 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.960179 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.961459 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.961474 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.961485 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.962114 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.963467 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966368 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966485 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966511 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966521 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966534 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966559 2580 kubelet.go:398] "Adding apiserver pod source" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966567 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 14:08:27.172016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966899 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.966915 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.968693 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.970829 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.970856 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972547 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972561 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972567 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972572 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972580 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972586 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972595 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972600 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972606 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972613 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972626 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972632 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972651 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.972656 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.975748 2580 server.go:1268] "Started kubelet" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.975822 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-km6tn" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.976017 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.976216 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.976255 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.978053 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.978127 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.988200 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-km6tn" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:26.988957 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:26.989205 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:26.989404 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:26.989134 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-213.ec2.internal.18beccd281a4be9e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-213.ec2.internal,UID:ip-10-0-132-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-213.ec2.internal,},FirstTimestamp:2026-07-03 14:08:26.975551134 +0000 UTC m=+0.397315604,LastTimestamp:2026-07-03 14:08:26.975551134 +0000 UTC m=+0.397315604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-213.ec2.internal,}" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.995048 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.995555 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.996550 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.996120 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.997042 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:26.997051 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:26.997106 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.004401 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.004409 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 03 14:08:27.189196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.004832 2580 factory.go:55] Registering systemd factory Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.004847 2580 factory.go:223] Registration of the systemd container factory successfully Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005037 2580 factory.go:153] Registering CRI-O factory Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005047 2580 factory.go:223] Registration of the crio container factory successfully Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005099 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005123 2580 factory.go:103] Registering Raw factory Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005137 2580 manager.go:1196] Started watching for new ooms in manager Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.005437 2580 manager.go:349] Starting recovery of all containers Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.015005 2580 manager.go:354] Recovery completed Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.016174 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.020323 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.021129 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-213.ec2.internal\" not found" node="ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022056 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022086 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022101 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022591 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022598 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022608 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.022614 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.023337 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.023362 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.023376 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.024293 2580 policy_none.go:49] "None policy: Start" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.024307 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.024320 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.025400 2580 policy_none.go:47] "Start" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055276 2580 manager.go:337] "Starting Device Plugin manager" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.055302 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055312 2580 server.go:87] "Starting device plugin registration server" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055564 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055575 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055719 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055822 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.055832 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.056540 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.056592 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.156928 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.158306 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.158333 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.158343 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.158370 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.168939 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.168970 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-213.ec2.internal\": node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.190509 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.185716 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.197706 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.197683 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 14:08:27.198816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.198795 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 14:08:27.198873 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.198843 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 14:08:27.198873 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.198869 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 14:08:27.198937 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.198907 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 14:08:27.200817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.200799 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:27.285886 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.285850 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.299099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.299075 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal"] Jul 03 14:08:27.299173 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.299139 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.300101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.300085 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.300176 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.300115 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.300176 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.300126 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.301481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.301468 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.301589 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.301572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.301639 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.301608 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.302241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302212 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.302241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302238 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.302241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302217 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.302453 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302248 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.302453 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302265 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.302453 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.302280 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.303373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.303355 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.303451 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.303387 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 14:08:27.304124 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.304108 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientMemory" Jul 03 14:08:27.304199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.304139 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 14:08:27.304199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.304156 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeHasSufficientPID" Jul 03 14:08:27.331584 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.331561 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-213.ec2.internal\" not found" node="ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.336213 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.336197 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-213.ec2.internal\" not found" node="ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.386320 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.386296 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.406582 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.406557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e2f5c65747e411c473e749cc46d5684-config\") pod \"kube-apiserver-proxy-ip-10-0-132-213.ec2.internal\" (UID: \"6e2f5c65747e411c473e749cc46d5684\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.406682 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.406589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.406682 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.406627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.487158 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.487113 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.506859 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e2f5c65747e411c473e749cc46d5684-config\") pod \"kube-apiserver-proxy-ip-10-0-132-213.ec2.internal\" (UID: \"6e2f5c65747e411c473e749cc46d5684\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.506981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.506981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.506981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e2f5c65747e411c473e749cc46d5684-config\") pod \"kube-apiserver-proxy-ip-10-0-132-213.ec2.internal\" (UID: \"6e2f5c65747e411c473e749cc46d5684\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.506981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506956 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.506981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.506971 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c75f48dab69a758b3baaeda081c26457-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal\" (UID: \"c75f48dab69a758b3baaeda081c26457\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.593659 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.593584 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.635178 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.635154 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.640463 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.640446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:27.694119 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.693949 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.795270 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.795225 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.880576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.880472 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 14:08:27.880723 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.880664 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 14:08:27.880723 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.880673 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 14:08:27.895733 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.895707 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.996545 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:27.996500 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:27.996695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.996522 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 14:03:27 +0000 UTC" deadline="2028-01-13 03:01:53.270436192 +0000 UTC" Jul 03 14:08:27.996695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:27.996567 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13404h53m25.273872206s" Jul 03 14:08:28.062983 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.062955 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:28.097637 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:28.097611 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-213.ec2.internal\" not found" Jul 03 14:08:28.152277 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.152250 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:28.197350 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.197323 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" Jul 03 14:08:28.212311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.212288 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 14:08:28.213273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.213257 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" Jul 03 14:08:28.226141 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.226121 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 14:08:28.229022 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:28.228993 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2f5c65747e411c473e749cc46d5684.slice/crio-5f49e119556a3a2561e1a7af37f8bec939b498b57923aa9c817c7f122ccc8666 WatchSource:0}: Error finding container 5f49e119556a3a2561e1a7af37f8bec939b498b57923aa9c817c7f122ccc8666: Status 404 returned error can't find the container with id 5f49e119556a3a2561e1a7af37f8bec939b498b57923aa9c817c7f122ccc8666 Jul 03 14:08:28.233442 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.233424 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:08:28.358854 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:28.358826 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75f48dab69a758b3baaeda081c26457.slice/crio-7e6cdb6a9d452f6ff0420ea38cb0567e94a15e3798872a5ac3fb1ace5d89b65e WatchSource:0}: Error finding container 7e6cdb6a9d452f6ff0420ea38cb0567e94a15e3798872a5ac3fb1ace5d89b65e: Status 404 returned error can't find the container with id 7e6cdb6a9d452f6ff0420ea38cb0567e94a15e3798872a5ac3fb1ace5d89b65e Jul 03 14:08:28.852303 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.852267 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:28.963192 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.963161 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 14:08:28.971494 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.971457 2580 apiserver.go:52] "Watching apiserver" Jul 03 14:08:28.979448 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.979092 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 14:08:28.979614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.979578 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dzgm9","kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal","openshift-dns/node-resolver-m4tvl","openshift-image-registry/node-ca-r6zkn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal","openshift-network-diagnostics/network-check-target-mvptm","openshift-network-operator/iptables-alerter-r2qkr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp","openshift-cluster-node-tuning-operator/tuned-svcrb","openshift-multus/multus-additional-cni-plugins-4b79s","openshift-multus/multus-llnq5","openshift-multus/network-metrics-daemon-bc6rt","openshift-ovn-kubernetes/ovnkube-node-btvqp","kube-system/global-pull-secret-syncer-xdtqp"] Jul 03 14:08:28.981363 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.981330 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:28.982453 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.982430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-llnq5" Jul 03 14:08:28.984390 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.984371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:28.985140 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.985093 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 14:08:28.985314 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.985292 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 14:08:28.985314 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.985304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 14:08:28.985450 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.985336 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 14:08:28.985450 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.985304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 14:08:28.988497 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.988470 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhkk2\"" Jul 03 14:08:28.988611 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.988560 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 14:08:28.989152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.988472 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lds9x\"" Jul 03 14:08:28.989152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.989125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 14:08:28.989535 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.989515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:28.991124 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.991096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:28.991296 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:28.991210 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:28.991517 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.991497 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 14:08:28.991796 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.991777 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 14:08:28.991935 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.991914 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mgdx7\"" Jul 03 14:08:28.992012 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.991995 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 14:08:28.992220 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.992202 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 14:08:28.993101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.993083 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:28.993246 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:28.993221 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:28.993699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.993678 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4jlm2\"" Jul 03 14:08:28.994045 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.994023 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 14:08:28.994340 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.994322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:28.998089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.997342 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 14:08:28.998089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.997354 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 14:08:28.998089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.997434 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7bkql\"" Jul 03 14:08:28.998089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.997696 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 14:08:28.998889 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:28.998870 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.000881 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.000509 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.003238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.003526 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.003742 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004387 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ngmrj\"" Jul 03 14:08:29.004481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004481 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 14:08:29.004974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004735 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 14:08:29.004974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 14:08:29.004974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004833 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 14:08:29.004974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004842 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 14:08:29.005286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.004997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.005476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.005455 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zhrms\"" Jul 03 14:08:29.005552 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.005496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 14:08:29.006256 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.005941 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.006256 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.006012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:29.008953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.008934 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 14:08:29.009373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.009354 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fsfl7\"" Jul 03 14:08:29.009482 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.009464 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dmrfl\"" Jul 03 14:08:29.009676 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.009647 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 14:08:29.009754 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.009711 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 14:08:29.009896 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.009855 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 14:08:29.014603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014577 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbh9m\" (UniqueName: \"kubernetes.io/projected/51bc5929-8271-48fe-96e3-b749edc4778a-kube-api-access-fbh9m\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.014697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-k8s-cni-cncf-io\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.014697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-hosts-file\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.014697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014679 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-kubelet-config\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysconfig\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-ovn\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovn-node-metrics-cert\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-script-lib\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51bc5929-8271-48fe-96e3-b749edc4778a-host-slash\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-socket-dir-parent\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.014876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgtnc\" (UniqueName: \"kubernetes.io/projected/7482ce98-1873-43c5-9f79-2f7e400f63d6-kube-api-access-tgtnc\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014898 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-sys\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.014972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-host\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-bin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-multus-daemon-config\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-systemd-units\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-bin\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-os-release\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-etc-kubernetes\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnz2s\" (UniqueName: \"kubernetes.io/projected/42f756a1-f1de-47ba-84de-146652646569-kube-api-access-rnz2s\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015208 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-systemd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-env-overrides\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51bc5929-8271-48fe-96e3-b749edc4778a-iptables-alerter-script\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-netns\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-hostroot\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-dbus\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-log-socket\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrhgt\" (UniqueName: \"kubernetes.io/projected/c2d86c2a-286f-4717-b132-6a3eea2efb56-kube-api-access-wrhgt\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-cnibin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-cni-binary-copy\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-conf-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-systemd\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-var-lib-kubelet\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-etc-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-config\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-multus-certs\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-kubernetes\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-multus\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.015898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-tmp\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015763 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-kubelet\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015812 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-run\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-node-log\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08296dc8-7d3d-405b-b524-3e13780e3081-host\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-modprobe-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-tuned\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.015987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-slash\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-netns\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-var-lib-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-tmp-dir\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.016737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-netd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-system-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q25jt\" (UniqueName: \"kubernetes.io/projected/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-kube-api-access-q25jt\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-conf\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-lib-modules\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-kubelet\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08296dc8-7d3d-405b-b524-3e13780e3081-serviceca\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.017748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.016411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mmj4\" (UniqueName: \"kubernetes.io/projected/08296dc8-7d3d-405b-b524-3e13780e3081-kube-api-access-9mmj4\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.027404 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.027358 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 14:08:29.044983 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.044887 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qnwk6" Jul 03 14:08:29.054521 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.054501 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qnwk6" Jul 03 14:08:29.098018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.097977 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 14:08:29.116850 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysconfig\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.116850 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-ovn\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovn-node-metrics-cert\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-script-lib\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51bc5929-8271-48fe-96e3-b749edc4778a-host-slash\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-socket-dir-parent\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgtnc\" (UniqueName: \"kubernetes.io/projected/7482ce98-1873-43c5-9f79-2f7e400f63d6-kube-api-access-tgtnc\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-sys\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-host\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117007 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysconfig\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.117041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-bin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-multus-daemon-config\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-agent-certs\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-socket-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-systemd-units\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-bin\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-os-release\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-etc-kubernetes\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnz2s\" (UniqueName: \"kubernetes.io/projected/42f756a1-f1de-47ba-84de-146652646569-kube-api-access-rnz2s\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-systemd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-env-overrides\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51bc5929-8271-48fe-96e3-b749edc4778a-iptables-alerter-script\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-netns\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.117481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-hostroot\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-dbus\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-log-socket\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrhgt\" (UniqueName: \"kubernetes.io/projected/c2d86c2a-286f-4717-b132-6a3eea2efb56-kube-api-access-wrhgt\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-cnibin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-cni-binary-copy\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-conf-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-konnectivity-ca\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-systemd\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117696 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-script-lib\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-var-lib-kubelet\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-etc-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-config\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117782 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51bc5929-8271-48fe-96e3-b749edc4778a-host-slash\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-multus-certs\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cnibin\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117860 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-multus-certs\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.118240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-kubernetes\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-multus\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-registration-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.117973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-device-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9xpb\" (UniqueName: \"kubernetes.io/projected/cbd0c789-3501-43a6-b5b9-8197f09fd167-kube-api-access-r9xpb\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4nz\" (UniqueName: \"kubernetes.io/projected/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-kube-api-access-tf4nz\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-tmp\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-kubelet\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.116954 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-ovn\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118203 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-kubernetes\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-sys\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118330 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-socket-dir-parent\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118499 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118830 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/51bc5929-8271-48fe-96e3-b749edc4778a-iptables-alerter-script\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118892 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-netns\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118891 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-multus-daemon-config\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118955 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-hostroot\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-dbus\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-log-socket\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.118145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119317 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-kubelet\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-sys-fs\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-etc-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-run\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119478 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-multus\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-node-log\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-var-lib-kubelet\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119415 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-conf-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08296dc8-7d3d-405b-b524-3e13780e3081-host\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119524 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-node-log\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-modprobe-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.119967 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-tuned\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-slash\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-netns\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119915 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovnkube-config\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-var-lib-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.119975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-systemd-units\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-tmp-dir\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120023 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-run\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08296dc8-7d3d-405b-b524-3e13780e3081-host\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-os-release\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-netd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120117 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-systemd\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-system-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120137 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42f756a1-f1de-47ba-84de-146652646569-cni-binary-copy\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.120783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120157 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-cnibin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q25jt\" (UniqueName: \"kubernetes.io/projected/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-kube-api-access-q25jt\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120272 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-systemd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-netd\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-cni-bin\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120321 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-host\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.120360 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-os-release\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-ovn-kubernetes\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-run-netns\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.120461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:29.62043819 +0000 UTC m=+3.042202650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-cni-bin\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-tmp-dir\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-etc-kubernetes\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120704 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-modprobe-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.121641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-conf\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-multus-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-lib-modules\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120787 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-kubelet\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-var-lib-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.120895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08296dc8-7d3d-405b-b524-3e13780e3081-serviceca\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121087 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-d\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mmj4\" (UniqueName: \"kubernetes.io/projected/08296dc8-7d3d-405b-b524-3e13780e3081-kube-api-access-9mmj4\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-system-cni-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbh9m\" (UniqueName: \"kubernetes.io/projected/51bc5929-8271-48fe-96e3-b749edc4778a-kube-api-access-fbh9m\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-k8s-cni-cncf-io\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-etc-selinux\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121313 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2d86c2a-286f-4717-b132-6a3eea2efb56-env-overrides\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsjxq\" (UniqueName: \"kubernetes.io/projected/d07d0072-3662-42cf-86b5-8af045828769-kube-api-access-lsjxq\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-hosts-file\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-kubelet-config\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.122507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121484 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-hosts-file\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d02fb523-8f63-44bb-8b13-c7cb33393387-kubelet-config\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121572 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-run-openvswitch\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-lib-modules\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121718 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08296dc8-7d3d-405b-b524-3e13780e3081-serviceca\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2d86c2a-286f-4717-b132-6a3eea2efb56-host-slash\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121819 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-var-lib-kubelet\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121855 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-sysctl-conf\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.121721 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-system-cni-dir\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.123384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.122037 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42f756a1-f1de-47ba-84de-146652646569-host-run-k8s-cni-cncf-io\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.126958 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.126932 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2d86c2a-286f-4717-b132-6a3eea2efb56-ovn-node-metrics-cert\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.128032 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.128013 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-tmp\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.128327 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.128301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7482ce98-1873-43c5-9f79-2f7e400f63d6-etc-tuned\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.129317 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.129295 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:29.129317 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.129322 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:29.129471 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.129336 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:29.129471 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.129393 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:29.62937442 +0000 UTC m=+3.051138898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:29.131065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.131010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgtnc\" (UniqueName: \"kubernetes.io/projected/7482ce98-1873-43c5-9f79-2f7e400f63d6-kube-api-access-tgtnc\") pod \"tuned-svcrb\" (UID: \"7482ce98-1873-43c5-9f79-2f7e400f63d6\") " pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.132013 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.131969 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnz2s\" (UniqueName: \"kubernetes.io/projected/42f756a1-f1de-47ba-84de-146652646569-kube-api-access-rnz2s\") pod \"multus-llnq5\" (UID: \"42f756a1-f1de-47ba-84de-146652646569\") " pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.133466 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.133441 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrhgt\" (UniqueName: \"kubernetes.io/projected/c2d86c2a-286f-4717-b132-6a3eea2efb56-kube-api-access-wrhgt\") pod \"ovnkube-node-btvqp\" (UID: \"c2d86c2a-286f-4717-b132-6a3eea2efb56\") " pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.139953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.139909 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q25jt\" (UniqueName: \"kubernetes.io/projected/f16f0567-1d4e-49f6-ad00-f4185dd5eef4-kube-api-access-q25jt\") pod \"node-resolver-m4tvl\" (UID: \"f16f0567-1d4e-49f6-ad00-f4185dd5eef4\") " pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.140818 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.140708 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbh9m\" (UniqueName: \"kubernetes.io/projected/51bc5929-8271-48fe-96e3-b749edc4778a-kube-api-access-fbh9m\") pod \"iptables-alerter-r2qkr\" (UID: \"51bc5929-8271-48fe-96e3-b749edc4778a\") " pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.141352 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.141332 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mmj4\" (UniqueName: \"kubernetes.io/projected/08296dc8-7d3d-405b-b524-3e13780e3081-kube-api-access-9mmj4\") pod \"node-ca-r6zkn\" (UID: \"08296dc8-7d3d-405b-b524-3e13780e3081\") " pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.202832 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.202780 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" event={"ID":"c75f48dab69a758b3baaeda081c26457","Type":"ContainerStarted","Data":"7e6cdb6a9d452f6ff0420ea38cb0567e94a15e3798872a5ac3fb1ace5d89b65e"} Jul 03 14:08:29.203965 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.203937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" event={"ID":"6e2f5c65747e411c473e749cc46d5684","Type":"ContainerStarted","Data":"5f49e119556a3a2561e1a7af37f8bec939b498b57923aa9c817c7f122ccc8666"} Jul 03 14:08:29.222252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-system-cni-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-etc-selinux\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222349 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-system-cni-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsjxq\" (UniqueName: \"kubernetes.io/projected/d07d0072-3662-42cf-86b5-8af045828769-kube-api-access-lsjxq\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222421 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-etc-selinux\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-agent-certs\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.222467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-socket-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222571 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-socket-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-konnectivity-ca\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cnibin\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-registration-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-device-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9xpb\" (UniqueName: \"kubernetes.io/projected/cbd0c789-3501-43a6-b5b9-8197f09fd167-kube-api-access-r9xpb\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.222849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4nz\" (UniqueName: \"kubernetes.io/projected/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-kube-api-access-tf4nz\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-sys-fs\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.222949 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223053 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-registration-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223172 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.223217 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-os-release\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223269 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.223288 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:29.723266927 +0000 UTC m=+3.145031388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-device-dir\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-konnectivity-ca\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223318 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-cnibin\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223343 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-os-release\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.223767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.223355 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cbd0c789-3501-43a6-b5b9-8197f09fd167-sys-fs\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.225315 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.225283 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70d8fc23-58f1-4a6b-aeab-9e485a2bf288-agent-certs\") pod \"konnectivity-agent-dzgm9\" (UID: \"70d8fc23-58f1-4a6b-aeab-9e485a2bf288\") " pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.240722 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.240697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsjxq\" (UniqueName: \"kubernetes.io/projected/d07d0072-3662-42cf-86b5-8af045828769-kube-api-access-lsjxq\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.243186 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.243160 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4nz\" (UniqueName: \"kubernetes.io/projected/e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa-kube-api-access-tf4nz\") pod \"multus-additional-cni-plugins-4b79s\" (UID: \"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa\") " pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.243747 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.243727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9xpb\" (UniqueName: \"kubernetes.io/projected/cbd0c789-3501-43a6-b5b9-8197f09fd167-kube-api-access-r9xpb\") pod \"aws-ebs-csi-driver-node-h74xp\" (UID: \"cbd0c789-3501-43a6-b5b9-8197f09fd167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.299828 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.299806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2qkr" Jul 03 14:08:29.309499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.309472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-llnq5" Jul 03 14:08:29.309914 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.309881 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bc5929_8271_48fe_96e3_b749edc4778a.slice/crio-daf3644cbdf254f8592dec478fea60490b9a2038ccdbb8b6a7a9124a7dd52be2 WatchSource:0}: Error finding container daf3644cbdf254f8592dec478fea60490b9a2038ccdbb8b6a7a9124a7dd52be2: Status 404 returned error can't find the container with id daf3644cbdf254f8592dec478fea60490b9a2038ccdbb8b6a7a9124a7dd52be2 Jul 03 14:08:29.318278 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.318250 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f756a1_f1de_47ba_84de_146652646569.slice/crio-3ff76b2204556fe1c4525c908bf14fff143e37ee02302c03f009eabde3a816d4 WatchSource:0}: Error finding container 3ff76b2204556fe1c4525c908bf14fff143e37ee02302c03f009eabde3a816d4: Status 404 returned error can't find the container with id 3ff76b2204556fe1c4525c908bf14fff143e37ee02302c03f009eabde3a816d4 Jul 03 14:08:29.320014 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.319778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4tvl" Jul 03 14:08:29.326938 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.326864 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r6zkn" Jul 03 14:08:29.329519 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.329487 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16f0567_1d4e_49f6_ad00_f4185dd5eef4.slice/crio-50759ba43e378b85ad5d59b6300c87748d14a1d12a4cd058ce68ca98817a0b95 WatchSource:0}: Error finding container 50759ba43e378b85ad5d59b6300c87748d14a1d12a4cd058ce68ca98817a0b95: Status 404 returned error can't find the container with id 50759ba43e378b85ad5d59b6300c87748d14a1d12a4cd058ce68ca98817a0b95 Jul 03 14:08:29.335689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.335645 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-svcrb" Jul 03 14:08:29.335957 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.335930 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08296dc8_7d3d_405b_b524_3e13780e3081.slice/crio-5c773fe3d6274bc29cfcbf80fdd5c6ee9dee407b8cbc17386d8e67f7c03398e1 WatchSource:0}: Error finding container 5c773fe3d6274bc29cfcbf80fdd5c6ee9dee407b8cbc17386d8e67f7c03398e1: Status 404 returned error can't find the container with id 5c773fe3d6274bc29cfcbf80fdd5c6ee9dee407b8cbc17386d8e67f7c03398e1 Jul 03 14:08:29.344093 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.344071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:29.344562 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.344541 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7482ce98_1873_43c5_9f79_2f7e400f63d6.slice/crio-c1c0de738e439d5d1dc735f5b4906c9c82fd59ef9e927f7fe1648e4707b31881 WatchSource:0}: Error finding container c1c0de738e439d5d1dc735f5b4906c9c82fd59ef9e927f7fe1648e4707b31881: Status 404 returned error can't find the container with id c1c0de738e439d5d1dc735f5b4906c9c82fd59ef9e927f7fe1648e4707b31881 Jul 03 14:08:29.353550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.353529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" Jul 03 14:08:29.354761 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.354720 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d86c2a_286f_4717_b132_6a3eea2efb56.slice/crio-786c14a0136b32e7b4153f24ee1190a0f1c6520e3cc2cc9bf89bc49cf86ccf00 WatchSource:0}: Error finding container 786c14a0136b32e7b4153f24ee1190a0f1c6520e3cc2cc9bf89bc49cf86ccf00: Status 404 returned error can't find the container with id 786c14a0136b32e7b4153f24ee1190a0f1c6520e3cc2cc9bf89bc49cf86ccf00 Jul 03 14:08:29.360937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.360916 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4b79s" Jul 03 14:08:29.363298 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.363272 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd0c789_3501_43a6_b5b9_8197f09fd167.slice/crio-264f724095563d98d7fa09516332e0686c9f567fc15b2999b4a18aa0d1292d11 WatchSource:0}: Error finding container 264f724095563d98d7fa09516332e0686c9f567fc15b2999b4a18aa0d1292d11: Status 404 returned error can't find the container with id 264f724095563d98d7fa09516332e0686c9f567fc15b2999b4a18aa0d1292d11 Jul 03 14:08:29.367803 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.367782 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:29.371703 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.371678 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08c0a3f_cb79_40b8_8587_ed2f2c02b7fa.slice/crio-7ce35a1a628d41defdcf2243dfea82e59a4fdc6a8470dc287207d543557a23d1 WatchSource:0}: Error finding container 7ce35a1a628d41defdcf2243dfea82e59a4fdc6a8470dc287207d543557a23d1: Status 404 returned error can't find the container with id 7ce35a1a628d41defdcf2243dfea82e59a4fdc6a8470dc287207d543557a23d1 Jul 03 14:08:29.380230 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:08:29.380205 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70d8fc23_58f1_4a6b_aeab_9e485a2bf288.slice/crio-9e3d0d76eab483f7cf293317a865c8b6c2a095e206a59845005803d910d301c9 WatchSource:0}: Error finding container 9e3d0d76eab483f7cf293317a865c8b6c2a095e206a59845005803d910d301c9: Status 404 returned error can't find the container with id 9e3d0d76eab483f7cf293317a865c8b6c2a095e206a59845005803d910d301c9 Jul 03 14:08:29.626402 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.626313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:29.626541 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.626435 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:29.626541 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.626517 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:30.62649328 +0000 UTC m=+4.048257757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:29.727643 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.727587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:29.727842 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:29.727661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:29.727842 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727755 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:29.727842 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727803 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:29.727842 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727828 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:29.727842 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727843 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:29.728124 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727831 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:30.727809239 +0000 UTC m=+4.149573699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:29.728124 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:29.727898 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:30.727885165 +0000 UTC m=+4.149649626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:30.058022 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.057906 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:03:29 +0000 UTC" deadline="2028-03-07 04:16:30.605774824 +0000 UTC" Jul 03 14:08:30.058022 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.057971 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14702h8m0.547808594s" Jul 03 14:08:30.200892 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.200063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:30.200892 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.200203 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:30.209750 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.209704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"786c14a0136b32e7b4153f24ee1190a0f1c6520e3cc2cc9bf89bc49cf86ccf00"} Jul 03 14:08:30.211920 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.211884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r6zkn" event={"ID":"08296dc8-7d3d-405b-b524-3e13780e3081","Type":"ContainerStarted","Data":"5c773fe3d6274bc29cfcbf80fdd5c6ee9dee407b8cbc17386d8e67f7c03398e1"} Jul 03 14:08:30.215774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.215705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-llnq5" event={"ID":"42f756a1-f1de-47ba-84de-146652646569","Type":"ContainerStarted","Data":"3ff76b2204556fe1c4525c908bf14fff143e37ee02302c03f009eabde3a816d4"} Jul 03 14:08:30.218777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.218749 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2qkr" event={"ID":"51bc5929-8271-48fe-96e3-b749edc4778a","Type":"ContainerStarted","Data":"daf3644cbdf254f8592dec478fea60490b9a2038ccdbb8b6a7a9124a7dd52be2"} Jul 03 14:08:30.222916 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.222839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerStarted","Data":"7ce35a1a628d41defdcf2243dfea82e59a4fdc6a8470dc287207d543557a23d1"} Jul 03 14:08:30.225724 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.225693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" event={"ID":"cbd0c789-3501-43a6-b5b9-8197f09fd167","Type":"ContainerStarted","Data":"264f724095563d98d7fa09516332e0686c9f567fc15b2999b4a18aa0d1292d11"} Jul 03 14:08:30.232679 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.232651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-svcrb" event={"ID":"7482ce98-1873-43c5-9f79-2f7e400f63d6","Type":"ContainerStarted","Data":"c1c0de738e439d5d1dc735f5b4906c9c82fd59ef9e927f7fe1648e4707b31881"} Jul 03 14:08:30.234295 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.234269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4tvl" event={"ID":"f16f0567-1d4e-49f6-ad00-f4185dd5eef4","Type":"ContainerStarted","Data":"50759ba43e378b85ad5d59b6300c87748d14a1d12a4cd058ce68ca98817a0b95"} Jul 03 14:08:30.236316 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.236290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dzgm9" event={"ID":"70d8fc23-58f1-4a6b-aeab-9e485a2bf288","Type":"ContainerStarted","Data":"9e3d0d76eab483f7cf293317a865c8b6c2a095e206a59845005803d910d301c9"} Jul 03 14:08:30.636716 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.636663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:30.636916 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.636900 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:30.637004 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.636972 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:32.636951748 +0000 UTC m=+6.058716213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:30.737199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.737114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:30.737199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:30.737213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:30.737457 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737381 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:30.737457 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737409 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:30.737457 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737424 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:30.737619 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737485 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:32.737465599 +0000 UTC m=+6.159230075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:30.737941 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737826 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:30.737941 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:30.737906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:32.737887046 +0000 UTC m=+6.159651531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:31.058343 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:31.058244 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:03:29 +0000 UTC" deadline="2027-12-28 01:23:35.130752672 +0000 UTC" Jul 03 14:08:31.058343 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:31.058291 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13019h15m4.07246812s" Jul 03 14:08:31.205192 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:31.204235 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:31.205192 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:31.204446 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:31.205192 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:31.205008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:31.205192 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:31.205141 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:32.199359 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:32.199267 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:32.199908 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.199432 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:32.654576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:32.654030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:32.654576 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.654219 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:32.654576 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.654281 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:36.654262058 +0000 UTC m=+10.076026535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:32.755490 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:32.755439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:32.755676 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:32.755522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:32.755810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755681 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:32.755810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755721 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:32.755810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755787 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:32.755810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755803 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:32.756032 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755763 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:36.755741758 +0000 UTC m=+10.177506220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:32.756032 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:32.755881 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:36.755853704 +0000 UTC m=+10.177618177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:33.202993 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:33.202217 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:33.202993 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:33.202356 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:33.202993 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:33.202748 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:33.202993 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:33.202864 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:34.200155 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:34.200053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:34.200339 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:34.200197 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:35.200941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:35.200077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:35.200941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:35.200074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:35.200941 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:35.200265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:35.200941 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:35.200334 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:36.199919 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:36.199880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:36.200119 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.200023 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:36.690134 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:36.690053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:36.690617 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.690188 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:36.690617 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.690253 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:44.690234912 +0000 UTC m=+18.111999391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:36.791349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:36.791437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.791581 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.791606 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.791619 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.791682 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:44.791661014 +0000 UTC m=+18.213425478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.791956 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:36.792083 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:36.792036 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:44.792012811 +0000 UTC m=+18.213777284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:37.200597 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:37.200563 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:37.200803 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:37.200698 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:37.200803 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:37.200719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:37.200948 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:37.200849 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:38.199776 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:38.199709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:38.200245 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:38.199866 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:39.200077 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:39.200040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:39.200501 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:39.200193 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:39.200501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:39.200252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:39.200501 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:39.200381 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:40.199612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:40.199161 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:40.199612 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:40.199283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:41.199846 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:41.199504 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:41.199846 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:41.199568 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:41.200258 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:41.199882 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:41.200258 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:41.199984 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:41.261598 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:41.261559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" event={"ID":"6e2f5c65747e411c473e749cc46d5684","Type":"ContainerStarted","Data":"fa088c4eb6672d8a9d80d2a9984868a1474c5ae6d77903cb019fbe0325067d3d"} Jul 03 14:08:41.278956 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:41.278911 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-213.ec2.internal" podStartSLOduration=12.278891914999999 podStartE2EDuration="12.278891915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:41.278585058 +0000 UTC m=+14.700349698" watchObservedRunningTime="2026-07-03 14:08:41.278891915 +0000 UTC m=+14.700656396" Jul 03 14:08:42.199207 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:42.199185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:42.199338 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:42.199298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:42.264701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:42.264667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" event={"ID":"cbd0c789-3501-43a6-b5b9-8197f09fd167","Type":"ContainerStarted","Data":"11f1e5ecde972cd92471d6a5e37d20e3f7fb45ffe173e0d2dd384d6d111d1146"} Jul 03 14:08:42.266535 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:42.266505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4tvl" event={"ID":"f16f0567-1d4e-49f6-ad00-f4185dd5eef4","Type":"ContainerStarted","Data":"3813eea72519961a812502cf7b68d67ba0559873615771da7e61fbbd0dc847c2"} Jul 03 14:08:42.268137 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:42.268116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" event={"ID":"c75f48dab69a758b3baaeda081c26457","Type":"ContainerStarted","Data":"0f145d6f55f6b67983ee2fbc424eede8b3b6f2cb82962299644da64f01b1db1e"} Jul 03 14:08:42.269802 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:42.269779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dzgm9" event={"ID":"70d8fc23-58f1-4a6b-aeab-9e485a2bf288","Type":"ContainerStarted","Data":"44a5d61e8035d3de0b86970119b6dcf4999b61893b63e92eea7a0191323cbd0c"} Jul 03 14:08:43.199465 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:43.199428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:43.199643 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:43.199561 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:43.199643 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:43.199621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:43.199810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:43.199783 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:43.288380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:43.288333 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dzgm9" podStartSLOduration=3.7344366559999997 podStartE2EDuration="15.288316877s" totalImagesPullingTime="11.553880221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:42.288986358 +0000 UTC m=+15.710750840" watchObservedRunningTime="2026-07-03 14:08:43.288316877 +0000 UTC m=+16.710081358" Jul 03 14:08:43.840602 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:43.840570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:43.909434 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:43.909387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:44.199888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:44.199819 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:44.200032 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.199945 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:44.753336 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:44.753305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:44.753808 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.753459 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:44.753808 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.753530 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:00.753511109 +0000 UTC m=+34.175275585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:08:44.854167 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:44.854139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:44.854301 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:44.854192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:44.854301 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854284 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:44.854377 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854320 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:08:44.854377 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854332 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:08:44.854377 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854341 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:44.854377 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:00.854326599 +0000 UTC m=+34.276091060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:08:44.854377 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:44.854375 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:00.854364434 +0000 UTC m=+34.276128893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:08:45.199611 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:45.199577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:45.199831 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:45.199577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:45.199831 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:45.199741 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:45.199831 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:45.199789 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:45.278030 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:45.277995 2580 generic.go:356] "Generic (PLEG): container finished" podID="c75f48dab69a758b3baaeda081c26457" containerID="0f145d6f55f6b67983ee2fbc424eede8b3b6f2cb82962299644da64f01b1db1e" exitCode=0 Jul 03 14:08:45.278211 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:45.278048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" event={"ID":"c75f48dab69a758b3baaeda081c26457","Type":"ContainerDied","Data":"0f145d6f55f6b67983ee2fbc424eede8b3b6f2cb82962299644da64f01b1db1e"} Jul 03 14:08:45.294757 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:45.294690 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m4tvl" podStartSLOduration=5.689688875 podStartE2EDuration="17.2946731s" totalImagesPullingTime="11.604984225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:43.304929979 +0000 UTC m=+16.726694456" watchObservedRunningTime="2026-07-03 14:08:45.2946731 +0000 UTC m=+18.716437575" Jul 03 14:08:46.199617 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:46.199588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:46.200078 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:46.199713 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:46.901264 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:46.901224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:46.901883 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:46.901862 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:46.983149 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:46.983113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:47.178890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:47.178799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:47.200622 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:47.200216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:47.200622 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:47.200319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:47.200622 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:47.200377 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:47.200622 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:47.200468 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:48.199741 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:48.199706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:48.199904 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:48.199841 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:48.403449 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:48.403407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:48.505131 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:48.505059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:48.573086 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:48.573059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:49.200091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.200052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:49.200091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.200099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:49.200332 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:49.200203 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:49.200381 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:49.200323 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:49.290932 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.290903 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-svcrb" event={"ID":"7482ce98-1873-43c5-9f79-2f7e400f63d6","Type":"ContainerStarted","Data":"ad7d298e3f626059127541c67e3417380b11eeb2c6b2a1accc95adc4a79a1262"} Jul 03 14:08:49.311598 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.311543 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-svcrb" podStartSLOduration=9.70605918 podStartE2EDuration="21.311525466s" totalImagesPullingTime="11.605466286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:49.310970524 +0000 UTC m=+22.732735029" watchObservedRunningTime="2026-07-03 14:08:49.311525466 +0000 UTC m=+22.733289947" Jul 03 14:08:49.441819 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.441757 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 14:08:49.852525 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.852288 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:49.852639 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.852622 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 03 14:08:49.853039 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.853019 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dzgm9" Jul 03 14:08:49.940552 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:49.940524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:50.064147 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.064116 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:50.068029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.067932 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T14:08:49.441777304Z","UUID":"7c98c9e4-26af-4e73-8b20-d99b1f04b4aa","Handler":null,"Name":"","Endpoint":""} Jul 03 14:08:50.069799 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.069778 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 14:08:50.069911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.069818 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 14:08:50.164531 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.164463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:50.199590 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.199559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:50.199751 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:50.199672 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:50.295844 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.295811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"ffa042450c681d2b47c61ca7097ecfdf36bcba41a51b42b52faeeeba65fbea4f"} Jul 03 14:08:50.296000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.295906 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"a44e1455a7d244aab866294a5d5995fe41b83715d47933bac5915ecd85213ae2"} Jul 03 14:08:50.296753 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.296293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"0df9b3e85a44850c9ed186f11036932dbab3f9f76984094b5753051973a871dc"} Jul 03 14:08:50.296753 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.296418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"5eb542547e41f3f5330fd9f38a37e25bebdd245d92a224d88fa17c0f7be2fcb3"} Jul 03 14:08:50.296753 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.296563 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"e36c213d5a65fca1446887f4c8bcd2622630eaf8a07e9f9100ede084119bdff1"} Jul 03 14:08:50.296753 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.296632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"dd4d0ccc9cd2ea22ef5b3fc3c95262ab40bbe15a8223b298df32d59e2645763c"} Jul 03 14:08:50.299854 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.299828 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r6zkn" event={"ID":"08296dc8-7d3d-405b-b524-3e13780e3081","Type":"ContainerStarted","Data":"edf97b0e672e155fe57334ccb2d57f097d47f0c13e27026b587e3688c2ea730c"} Jul 03 14:08:50.301239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.301217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-llnq5" event={"ID":"42f756a1-f1de-47ba-84de-146652646569","Type":"ContainerStarted","Data":"f22f961f5f5493096da1428f5a9d6a0fa4cef048b8b1cbccfd323da9218feced"} Jul 03 14:08:50.302505 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.302483 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="a553fb8e7e93509cf3c49bf9964a057d91047874a674d0eec7a4c20c79d52309" exitCode=0 Jul 03 14:08:50.302592 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.302510 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"a553fb8e7e93509cf3c49bf9964a057d91047874a674d0eec7a4c20c79d52309"} Jul 03 14:08:50.304337 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.304321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" event={"ID":"cbd0c789-3501-43a6-b5b9-8197f09fd167","Type":"ContainerStarted","Data":"6f1226e46b47baef7460140f9697ce972237ace9daaeca38efa2eb897d987581"} Jul 03 14:08:50.305885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.305860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" event={"ID":"c75f48dab69a758b3baaeda081c26457","Type":"ContainerStarted","Data":"19ef773c5b7e810466e56c6c7a1c7bd01a4f534feb3b1e1af3a1a9e45ab24230"} Jul 03 14:08:50.318061 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.318025 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r6zkn" podStartSLOduration=10.695749642 podStartE2EDuration="22.318012551s" totalImagesPullingTime="11.622262909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:50.317435182 +0000 UTC m=+23.739199662" watchObservedRunningTime="2026-07-03 14:08:50.318012551 +0000 UTC m=+23.739777031" Jul 03 14:08:50.359221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.359176 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-213.ec2.internal" podStartSLOduration=21.359161426 podStartE2EDuration="21.359161426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:50.359081226 +0000 UTC m=+23.780845708" watchObservedRunningTime="2026-07-03 14:08:50.359161426 +0000 UTC m=+23.780925907" Jul 03 14:08:50.384331 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.384279 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-llnq5" podStartSLOduration=2.391668826 podStartE2EDuration="22.384264412s" totalImagesPullingTime="19.992595586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:50.38383179 +0000 UTC m=+23.805596270" watchObservedRunningTime="2026-07-03 14:08:50.384264412 +0000 UTC m=+23.806029081" Jul 03 14:08:50.548170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.548103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:50.630420 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:50.630392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:51.199231 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.199199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:51.199496 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:51.199307 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:51.199496 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.199379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:51.199601 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:51.199516 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:51.310119 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.310083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2qkr" event={"ID":"51bc5929-8271-48fe-96e3-b749edc4778a","Type":"ContainerStarted","Data":"f73cf612f5c530615ebe54b962e2a9b8e582b5e5c6ce2794f27a96e55106ce25"} Jul 03 14:08:51.312402 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.312376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" event={"ID":"cbd0c789-3501-43a6-b5b9-8197f09fd167","Type":"ContainerStarted","Data":"2b03e0a72cf2a04a3166c78bb7bcd349915c0aab0035aed997ac723f0d9194a1"} Jul 03 14:08:51.326301 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.326262 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r2qkr" podStartSLOduration=11.701008665 podStartE2EDuration="23.326250966s" totalImagesPullingTime="11.625242301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:51.326148965 +0000 UTC m=+24.747913448" watchObservedRunningTime="2026-07-03 14:08:51.326250966 +0000 UTC m=+24.748015447" Jul 03 14:08:51.344931 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.344890 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h74xp" podStartSLOduration=3.543882886 podStartE2EDuration="23.34487663s" totalImagesPullingTime="19.800993744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:51.344503817 +0000 UTC m=+24.766268310" watchObservedRunningTime="2026-07-03 14:08:51.34487663 +0000 UTC m=+24.766641134" Jul 03 14:08:51.468948 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.468861 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:51.538334 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:51.538291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:08:52.199560 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:52.199527 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:52.199969 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:52.199628 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:52.317259 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:52.317229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"d94a5569b6eec79c8b9d56d451625c1cf09f6f5c7903c42f638c17cd802040f4"} Jul 03 14:08:53.199091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:53.199058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:53.199280 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:53.199185 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:53.199280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:53.199248 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:53.199398 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:53.199370 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:54.200037 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:54.200008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:54.200477 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:54.200119 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:55.199660 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:55.199628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:55.199846 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:55.199760 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:55.199928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:55.199843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:55.199982 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:55.199960 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:56.199068 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.199035 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:56.199752 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:56.199132 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:56.327995 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.327964 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="1b0ab871ab5d33746d5828f586341be180c6b7e79d895fc6a25b1c66a3a122fc" exitCode=0 Jul 03 14:08:56.328137 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.328015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"1b0ab871ab5d33746d5828f586341be180c6b7e79d895fc6a25b1c66a3a122fc"} Jul 03 14:08:56.331714 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.331684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" event={"ID":"c2d86c2a-286f-4717-b132-6a3eea2efb56","Type":"ContainerStarted","Data":"656ce3d669ade78cef81f0f47457c581104831f0358f0ccc9fa51648c2136a4a"} Jul 03 14:08:56.332040 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.332017 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:56.347151 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.347132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:56.382709 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:56.382490 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" podStartSLOduration=8.398650325 podStartE2EDuration="28.382475918s" totalImagesPullingTime="19.983825593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:56.38204517 +0000 UTC m=+29.803809651" watchObservedRunningTime="2026-07-03 14:08:56.382475918 +0000 UTC m=+29.804240399" Jul 03 14:08:57.202237 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.201551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:57.202237 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:57.201675 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:57.202237 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.202099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:57.202237 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:57.202196 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:57.337013 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.336822 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="9f4c29bb279c98d95305627b3522f681e3da8f0f71909fa129c8c9cd7f348475" exitCode=0 Jul 03 14:08:57.337013 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.336901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"9f4c29bb279c98d95305627b3522f681e3da8f0f71909fa129c8c9cd7f348475"} Jul 03 14:08:57.337612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.337497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:57.337612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.337568 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:57.353668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.353643 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:08:57.542233 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.542133 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xdtqp"] Jul 03 14:08:57.542233 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.542231 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:57.542404 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:57.542306 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:08:57.545673 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.545648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bc6rt"] Jul 03 14:08:57.545820 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.545778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:57.545899 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:57.545880 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:57.546501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.546474 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mvptm"] Jul 03 14:08:57.546576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:57.546562 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:57.546675 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:57.546650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:58.341634 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:58.341603 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="eafee2f46299d8908c73f808821a2d3dc0d26bf3eb0ff8982b23013b6b93fd0a" exitCode=0 Jul 03 14:08:58.342051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:58.341674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"eafee2f46299d8908c73f808821a2d3dc0d26bf3eb0ff8982b23013b6b93fd0a"} Jul 03 14:08:59.200039 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:59.199798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:08:59.200195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:59.199804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:08:59.200195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:08:59.199828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:08:59.200195 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:59.200163 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:08:59.200347 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:59.200251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:08:59.200411 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:08:59.200386 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:09:00.770218 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:00.770183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:00.770588 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.770292 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 14:09:00.770588 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.770344 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret podName:d02fb523-8f63-44bb-8b13-c7cb33393387 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:32.770330204 +0000 UTC m=+66.192094662 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret") pod "global-pull-secret-syncer-xdtqp" (UID: "d02fb523-8f63-44bb-8b13-c7cb33393387") : object "kube-system"/"original-pull-secret" not registered Jul 03 14:09:00.871234 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:00.871156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:00.871234 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:00.871207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871309 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871318 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871337 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871349 2580 projected.go:196] Error preparing data for projected volume kube-api-access-fjbmx for pod openshift-network-diagnostics/network-check-target-mvptm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871374 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:32.871355181 +0000 UTC m=+66.293119641 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 14:09:00.871412 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:00.871391 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx podName:27d773c8-a973-4519-b705-dcae305e2294 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:32.871382082 +0000 UTC m=+66.293146546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fjbmx" (UniqueName: "kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx") pod "network-check-target-mvptm" (UID: "27d773c8-a973-4519-b705-dcae305e2294") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 14:09:01.200024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.199953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:01.200024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.199964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:01.200024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.199981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:09:01.200279 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:01.200059 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xdtqp" podUID="d02fb523-8f63-44bb-8b13-c7cb33393387" Jul 03 14:09:01.200279 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:01.200124 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvptm" podUID="27d773c8-a973-4519-b705-dcae305e2294" Jul 03 14:09:01.200279 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:01.200203 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bc6rt" podUID="d07d0072-3662-42cf-86b5-8af045828769" Jul 03 14:09:01.805555 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.805523 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-213.ec2.internal" event="NodeReady" Jul 03 14:09:01.806048 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.805660 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 14:09:01.869942 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.869913 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j74ld"] Jul 03 14:09:01.915760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.915707 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pkcmz"] Jul 03 14:09:01.915915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.915828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:01.918380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.918359 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wf6pg\"" Jul 03 14:09:01.918506 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.918404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 14:09:01.918757 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.918743 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 14:09:01.938694 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.938677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j74ld"] Jul 03 14:09:01.938794 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.938707 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkcmz"] Jul 03 14:09:01.938838 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.938806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:01.941845 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.941827 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 14:09:01.941941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.941870 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 14:09:01.941941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.941901 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 14:09:01.942029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:01.941948 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7fg4h\"" Jul 03 14:09:02.072652 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.072626 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:02.079791 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-cert\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.079917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d89c1a98-bdb5-4a93-972b-dce4073fbf34-config-volume\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.079917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89c1a98-bdb5-4a93-972b-dce4073fbf34-tmp-dir\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.079917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fh7s\" (UniqueName: \"kubernetes.io/projected/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-kube-api-access-2fh7s\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.080018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6pvr\" (UniqueName: \"kubernetes.io/projected/d89c1a98-bdb5-4a93-972b-dce4073fbf34-kube-api-access-b6pvr\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.080018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.079940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.147601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.147575 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:02.180628 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6pvr\" (UniqueName: \"kubernetes.io/projected/d89c1a98-bdb5-4a93-972b-dce4073fbf34-kube-api-access-b6pvr\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-cert\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d89c1a98-bdb5-4a93-972b-dce4073fbf34-config-volume\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89c1a98-bdb5-4a93-972b-dce4073fbf34-tmp-dir\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:02.180785 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:02.180801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.180794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh7s\" (UniqueName: \"kubernetes.io/projected/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-kube-api-access-2fh7s\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.181155 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:02.180846 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:02.680827399 +0000 UTC m=+36.102591877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:02.181226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.181198 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d89c1a98-bdb5-4a93-972b-dce4073fbf34-tmp-dir\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.181226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.181213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d89c1a98-bdb5-4a93-972b-dce4073fbf34-config-volume\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.185588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.185566 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-cert\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.190623 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.190588 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fh7s\" (UniqueName: \"kubernetes.io/projected/5f47ef9d-ee06-4552-877d-fcc4ebc14b8c-kube-api-access-2fh7s\") pod \"ingress-canary-pkcmz\" (UID: \"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c\") " pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.198256 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.198230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6pvr\" (UniqueName: \"kubernetes.io/projected/d89c1a98-bdb5-4a93-972b-dce4073fbf34-kube-api-access-b6pvr\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.246976 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.246945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkcmz" Jul 03 14:09:02.685261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:02.685226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:02.685436 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:02.685377 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:02.685485 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:02.685441 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:03.685425526 +0000 UTC m=+37.107189984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:03.202956 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.202907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:09:03.203631 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.203005 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:03.203631 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.203034 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:03.205701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.205679 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 14:09:03.206513 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.206494 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-472cc\"" Jul 03 14:09:03.206624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.206518 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 14:09:03.206624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.206526 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 14:09:03.206624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.206518 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 14:09:03.206624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.206618 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlbvf\"" Jul 03 14:09:03.692829 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.692800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:03.693011 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:03.692934 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:03.693011 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:03.692992 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:05.69297574 +0000 UTC m=+39.114740199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:03.824901 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:03.824694 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkcmz"] Jul 03 14:09:03.892685 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:03.892641 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f47ef9d_ee06_4552_877d_fcc4ebc14b8c.slice/crio-746bfded641ed2d8cbda4d421d0cfedc5acdf3795e9a9570e99108aa6fbfb9ba WatchSource:0}: Error finding container 746bfded641ed2d8cbda4d421d0cfedc5acdf3795e9a9570e99108aa6fbfb9ba: Status 404 returned error can't find the container with id 746bfded641ed2d8cbda4d421d0cfedc5acdf3795e9a9570e99108aa6fbfb9ba Jul 03 14:09:04.358813 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:04.358777 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="e5709da002a4a59f0e4fcaf4df7dad5fa486e9f9ae213fd28e4dd9617d38702d" exitCode=0 Jul 03 14:09:04.359407 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:04.358844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"e5709da002a4a59f0e4fcaf4df7dad5fa486e9f9ae213fd28e4dd9617d38702d"} Jul 03 14:09:04.360095 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:04.360076 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkcmz" event={"ID":"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c","Type":"ContainerStarted","Data":"746bfded641ed2d8cbda4d421d0cfedc5acdf3795e9a9570e99108aa6fbfb9ba"} Jul 03 14:09:04.572513 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:04.572467 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:04.673425 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:04.673347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:05.365624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:05.365592 2580 generic.go:356] "Generic (PLEG): container finished" podID="e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa" containerID="11ffcbabd29995fecd1fbf7e4e33a1ce25b44537b87453335653649a88b9ca6f" exitCode=0 Jul 03 14:09:05.366101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:05.365633 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerDied","Data":"11ffcbabd29995fecd1fbf7e4e33a1ce25b44537b87453335653649a88b9ca6f"} Jul 03 14:09:05.538648 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:05.538608 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:05.644304 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:05.644217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:05.709268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:05.709235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:05.709436 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:05.709366 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:05.709436 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:05.709431 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:09.709412982 +0000 UTC m=+43.131177455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:06.371659 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.371621 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4b79s" event={"ID":"e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa","Type":"ContainerStarted","Data":"e6d70cf9b3c820b310d806061a5012e2c306ea300b7ef2f2f3f62916121eaa70"} Jul 03 14:09:06.396147 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.396080 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4b79s" podStartSLOduration=14.428084386 podStartE2EDuration="38.396047995s" totalImagesPullingTime="23.967963609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:06.394910751 +0000 UTC m=+39.816675231" watchObservedRunningTime="2026-07-03 14:09:06.396047995 +0000 UTC m=+39.817812479" Jul 03 14:09:06.521701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.521674 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:06.586927 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.586887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:06.649410 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.649344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:06.885606 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.885577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:06.987162 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:06.987134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:07.205119 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:07.204911 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:07.307361 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:07.307329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:07.375353 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:07.375325 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkcmz" event={"ID":"5f47ef9d-ee06-4552-877d-fcc4ebc14b8c","Type":"ContainerStarted","Data":"92b258db80c532d68040aedc52c68b45f46c11c2d578e3a4e965b1247cbd8d51"} Jul 03 14:09:07.705884 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:07.705852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:07.793807 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:07.793769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:08.088578 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:08.088532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:08.154426 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:08.154400 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:08.579879 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:08.579840 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:09.741056 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:09.741013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:09.741408 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:09.741174 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:09.741408 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:09.741235 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:17.741219745 +0000 UTC m=+51.162984204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:17.152011 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.151960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:17.258299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.258266 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:17.794979 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.794943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:17.795219 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:17.795092 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:17.795219 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:17.795155 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:33.795139125 +0000 UTC m=+67.216903588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:17.931666 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.931609 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pkcmz" podStartSLOduration=13.945715801 podStartE2EDuration="16.931593103s" totalImagesPullingTime="2.985877302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:07.391409054 +0000 UTC m=+40.813173531" watchObservedRunningTime="2026-07-03 14:09:17.931593103 +0000 UTC m=+51.353357613" Jul 03 14:09:17.931866 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.931792 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ccctb"] Jul 03 14:09:17.941399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.941378 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:17.943996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.943971 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 14:09:17.943996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.943988 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 14:09:17.944134 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.944049 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r4m9l\"" Jul 03 14:09:17.944739 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.944708 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 03 14:09:17.944845 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.944749 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 03 14:09:17.950113 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.950092 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ccctb"] Jul 03 14:09:17.951913 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:17.951896 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 03 14:09:18.051166 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.051090 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh"] Jul 03 14:09:18.057593 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.057573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" Jul 03 14:09:18.061262 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.061241 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 03 14:09:18.061504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.061487 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 03 14:09:18.061591 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.061561 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t6hhw\"" Jul 03 14:09:18.071673 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.071652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh"] Jul 03 14:09:18.096981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.096956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.096981 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.096983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.097212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.097005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bde77271-76bc-428a-a4c4-f096114b6526-serving-cert\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.097212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.097032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-snapshots\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.097212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.097050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww68r\" (UniqueName: \"kubernetes.io/projected/bde77271-76bc-428a-a4c4-f096114b6526-kube-api-access-ww68r\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.097212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.097157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-tmp\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.144601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.144570 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:09:18.157737 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.157708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.163777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.163717 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:09:18.165676 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.165656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 03 14:09:18.166020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.166003 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 03 14:09:18.166094 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.166052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 03 14:09:18.166612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.166590 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-scvmw\"" Jul 03 14:09:18.173415 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.173400 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 03 14:09:18.198072 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-tmp\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bde77271-76bc-428a-a4c4-f096114b6526-serving-cert\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-snapshots\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198397 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww68r\" (UniqueName: \"kubernetes.io/projected/bde77271-76bc-428a-a4c4-f096114b6526-kube-api-access-ww68r\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198397 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84b66\" (UniqueName: \"kubernetes.io/projected/641f01c7-e593-41c8-bf48-9f1e8d4c24e7-kube-api-access-84b66\") pod \"volume-data-source-validator-5d456fdbcd-fqsxh\" (UID: \"641f01c7-e593-41c8-bf48-9f1e8d4c24e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" Jul 03 14:09:18.198524 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-tmp\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198786 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198766 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bde77271-76bc-428a-a4c4-f096114b6526-snapshots\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.198998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.198979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.199242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.199227 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde77271-76bc-428a-a4c4-f096114b6526-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.202326 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.202302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bde77271-76bc-428a-a4c4-f096114b6526-serving-cert\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.213513 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.213488 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww68r\" (UniqueName: \"kubernetes.io/projected/bde77271-76bc-428a-a4c4-f096114b6526-kube-api-access-ww68r\") pod \"insights-operator-5c847dcfc4-ccctb\" (UID: \"bde77271-76bc-428a-a4c4-f096114b6526\") " pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.250646 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.250628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" Jul 03 14:09:18.283683 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.283657 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:18.298813 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlgdn\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.298928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.298928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298842 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.298928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.298928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.299130 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.299130 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.298978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84b66\" (UniqueName: \"kubernetes.io/projected/641f01c7-e593-41c8-bf48-9f1e8d4c24e7-kube-api-access-84b66\") pod \"volume-data-source-validator-5d456fdbcd-fqsxh\" (UID: \"641f01c7-e593-41c8-bf48-9f1e8d4c24e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" Jul 03 14:09:18.299130 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.299000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.299130 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.299105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.308240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.308190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84b66\" (UniqueName: \"kubernetes.io/projected/641f01c7-e593-41c8-bf48-9f1e8d4c24e7-kube-api-access-84b66\") pod \"volume-data-source-validator-5d456fdbcd-fqsxh\" (UID: \"641f01c7-e593-41c8-bf48-9f1e8d4c24e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" Jul 03 14:09:18.366051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.366019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" Jul 03 14:09:18.387774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.387720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:18.388193 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.388170 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ccctb"] Jul 03 14:09:18.391103 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:18.391080 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde77271_76bc_428a_a4c4_f096114b6526.slice/crio-1246181467ab6af66efa9797e538fdf5a75d46d0f65e1417907131c78f3464d8 WatchSource:0}: Error finding container 1246181467ab6af66efa9797e538fdf5a75d46d0f65e1417907131c78f3464d8: Status 404 returned error can't find the container with id 1246181467ab6af66efa9797e538fdf5a75d46d0f65e1417907131c78f3464d8 Jul 03 14:09:18.399373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgdn\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399711 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.399711 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.399611 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:18.399711 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.399632 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:18.399906 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.399770 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:18.899746143 +0000 UTC m=+52.321510609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:18.399906 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.400020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.399935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.400076 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.400033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.400782 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.400711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" event={"ID":"bde77271-76bc-428a-a4c4-f096114b6526","Type":"ContainerStarted","Data":"1246181467ab6af66efa9797e538fdf5a75d46d0f65e1417907131c78f3464d8"} Jul 03 14:09:18.400876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.400823 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.402284 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.402266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.402424 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.402403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.408344 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.408326 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.408421 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.408361 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgdn\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.496280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.496242 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh"] Jul 03 14:09:18.499766 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:18.499740 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641f01c7_e593_41c8_bf48_9f1e8d4c24e7.slice/crio-f107eb02001448cec24c96f848f16bf58c778a12bd69702ec6e81fe1eb26e496 WatchSource:0}: Error finding container f107eb02001448cec24c96f848f16bf58c778a12bd69702ec6e81fe1eb26e496: Status 404 returned error can't find the container with id f107eb02001448cec24c96f848f16bf58c778a12bd69702ec6e81fe1eb26e496 Jul 03 14:09:18.907758 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:18.907705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:18.907939 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.907852 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:18.907939 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.907871 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:18.907939 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:18.907936 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:19.907919599 +0000 UTC m=+53.329684061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:19.406547 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:19.406320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" event={"ID":"641f01c7-e593-41c8-bf48-9f1e8d4c24e7","Type":"ContainerStarted","Data":"f107eb02001448cec24c96f848f16bf58c778a12bd69702ec6e81fe1eb26e496"} Jul 03 14:09:19.914511 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:19.914469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:19.914694 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:19.914632 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:19.914694 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:19.914653 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:19.914837 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:19.914741 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:21.914701948 +0000 UTC m=+55.336466423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:21.413334 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.413299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" event={"ID":"bde77271-76bc-428a-a4c4-f096114b6526","Type":"ContainerStarted","Data":"2ad443387b29fa5b699ce58280a1c712e2d822a568e474424f196955c5af6672"} Jul 03 14:09:21.414742 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.414705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" event={"ID":"641f01c7-e593-41c8-bf48-9f1e8d4c24e7","Type":"ContainerStarted","Data":"b957cf134e043aa6bcc360bdb7cb1d44ff69dd664c25bf4639bbf5d497389ed0"} Jul 03 14:09:21.444615 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.444559 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" podStartSLOduration=2.251028212 podStartE2EDuration="4.444539528s" totalImagesPullingTime="2.193511316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:21.444049124 +0000 UTC m=+54.865813607" watchObservedRunningTime="2026-07-03 14:09:21.444539528 +0000 UTC m=+54.866304010" Jul 03 14:09:21.481293 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.481244 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-fqsxh" podStartSLOduration=1.400263871 podStartE2EDuration="3.481229959s" totalImagesPullingTime="2.080966088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:21.480172404 +0000 UTC m=+54.901936885" watchObservedRunningTime="2026-07-03 14:09:21.481229959 +0000 UTC m=+54.902994439" Jul 03 14:09:21.658992 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.658949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:21.868582 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.868550 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:21.873384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.873357 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4s4gl"] Jul 03 14:09:21.875780 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.875765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.880047 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.880026 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 14:09:21.880496 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.880476 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 14:09:21.880874 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.880858 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kwmd4\"" Jul 03 14:09:21.901202 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.901176 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4s4gl"] Jul 03 14:09:21.929719 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.929691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/462292be-e44f-4b52-9608-0cb0207499f9-crio-socket\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.929862 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.929742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/462292be-e44f-4b52-9608-0cb0207499f9-data-volume\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.929945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.929891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:21.929945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.929918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.930029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.929959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.930029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:21.930010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k2fs\" (UniqueName: \"kubernetes.io/projected/462292be-e44f-4b52-9608-0cb0207499f9-kube-api-access-4k2fs\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:21.930029 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:21.930023 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:21.930129 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:21.930038 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:21.930129 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:21.930083 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.930065005 +0000 UTC m=+59.351829464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:22.030400 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k2fs\" (UniqueName: \"kubernetes.io/projected/462292be-e44f-4b52-9608-0cb0207499f9-kube-api-access-4k2fs\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/462292be-e44f-4b52-9608-0cb0207499f9-crio-socket\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/462292be-e44f-4b52-9608-0cb0207499f9-data-volume\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030507 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/462292be-e44f-4b52-9608-0cb0207499f9-crio-socket\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.030810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.030592 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:22.030810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.030645 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:22.530627946 +0000 UTC m=+55.952392409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:22.030810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.030648 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 14:09:22.030810 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.030699 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:22.530686038 +0000 UTC m=+55.952450499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : configmap "kube-rbac-proxy" not found Jul 03 14:09:22.030943 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.030845 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/462292be-e44f-4b52-9608-0cb0207499f9-data-volume\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.042943 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.042920 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k2fs\" (UniqueName: \"kubernetes.io/projected/462292be-e44f-4b52-9608-0cb0207499f9-kube-api-access-4k2fs\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.124903 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.124824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:22.434992 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.434917 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:22.534578 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.534549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.534767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.534589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:22.534767 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.534669 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 14:09:22.534767 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.534716 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:22.534767 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.534751 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:23.534717566 +0000 UTC m=+56.956482026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : configmap "kube-rbac-proxy" not found Jul 03 14:09:22.534966 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:22.534778 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:23.534765388 +0000 UTC m=+56.956529846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:22.535522 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:22.535503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:23.169552 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.169521 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:23.180952 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.180928 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7"] Jul 03 14:09:23.186040 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.186017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" Jul 03 14:09:23.188494 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.188473 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 03 14:09:23.188595 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.188549 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4sx6j\"" Jul 03 14:09:23.189227 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.189213 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 03 14:09:23.204824 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.204799 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7"] Jul 03 14:09:23.239248 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.239222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjs6\" (UniqueName: \"kubernetes.io/projected/dd7502f4-d10f-4e60-a667-53f1ec7c89ff-kube-api-access-sqjs6\") pod \"migrator-d57d58cc9-nnbp7\" (UID: \"dd7502f4-d10f-4e60-a667-53f1ec7c89ff\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" Jul 03 14:09:23.284532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.284498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:23.339926 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.339898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjs6\" (UniqueName: \"kubernetes.io/projected/dd7502f4-d10f-4e60-a667-53f1ec7c89ff-kube-api-access-sqjs6\") pod \"migrator-d57d58cc9-nnbp7\" (UID: \"dd7502f4-d10f-4e60-a667-53f1ec7c89ff\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" Jul 03 14:09:23.356308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.356284 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjs6\" (UniqueName: \"kubernetes.io/projected/dd7502f4-d10f-4e60-a667-53f1ec7c89ff-kube-api-access-sqjs6\") pod \"migrator-d57d58cc9-nnbp7\" (UID: \"dd7502f4-d10f-4e60-a667-53f1ec7c89ff\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" Jul 03 14:09:23.469210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.469144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:23.494919 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.494895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" Jul 03 14:09:23.542263 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.542230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:23.542363 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.542289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:23.542495 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:23.542471 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 14:09:23.542609 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:23.542518 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:23.542609 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:23.542535 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.542519103 +0000 UTC m=+58.964283567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : configmap "kube-rbac-proxy" not found Jul 03 14:09:23.542609 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:23.542571 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.54255446 +0000 UTC m=+58.964318943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:23.614108 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:23.614071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7"] Jul 03 14:09:23.617759 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:23.617716 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7502f4_d10f_4e60_a667_53f1ec7c89ff.slice/crio-5fb8455d07c615d47e8b53aac63175da9c2688b2978659bf8165c1e5eecb9692 WatchSource:0}: Error finding container 5fb8455d07c615d47e8b53aac63175da9c2688b2978659bf8165c1e5eecb9692: Status 404 returned error can't find the container with id 5fb8455d07c615d47e8b53aac63175da9c2688b2978659bf8165c1e5eecb9692 Jul 03 14:09:24.001923 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:24.001899 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4tvl_f16f0567-1d4e-49f6-ad00-f4185dd5eef4/dns-node-resolver/0.log" Jul 03 14:09:24.422946 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:24.422912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" event={"ID":"dd7502f4-d10f-4e60-a667-53f1ec7c89ff","Type":"ContainerStarted","Data":"5fb8455d07c615d47e8b53aac63175da9c2688b2978659bf8165c1e5eecb9692"} Jul 03 14:09:25.000817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.000794 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r6zkn_08296dc8-7d3d-405b-b524-3e13780e3081/node-ca/0.log" Jul 03 14:09:25.401759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.401704 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pkcmz_5f47ef9d-ee06-4552-877d-fcc4ebc14b8c/serve-healthcheck-canary/0.log" Jul 03 14:09:25.427280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.427250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" event={"ID":"dd7502f4-d10f-4e60-a667-53f1ec7c89ff","Type":"ContainerStarted","Data":"325f97d7e37e34e46078b7d0fed79dd1f11cbc76cc74c177ef8a6b405f429afb"} Jul 03 14:09:25.427404 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.427297 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" event={"ID":"dd7502f4-d10f-4e60-a667-53f1ec7c89ff","Type":"ContainerStarted","Data":"16afb4b0b138c3530ae8227aaa72bc5c682d55b90687b1c9719c9829e91d4f35"} Jul 03 14:09:25.446187 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.446038 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-nnbp7" podStartSLOduration=1.294212288 podStartE2EDuration="2.446024125s" totalImagesPullingTime="1.151811837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:25.445971105 +0000 UTC m=+58.867735603" watchObservedRunningTime="2026-07-03 14:09:25.446024125 +0000 UTC m=+58.867788622" Jul 03 14:09:25.549663 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.549631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:25.557272 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.557250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:25.557356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.557288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:25.557422 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.557381 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:25.557422 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.557384 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 14:09:25.557516 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.557427 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:29.557415426 +0000 UTC m=+62.979179885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:25.557516 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.557440 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:29.557434267 +0000 UTC m=+62.979198726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : configmap "kube-rbac-proxy" not found Jul 03 14:09:25.621782 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.621749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:25.722994 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.722915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:25.838029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.838003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:25.960651 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:25.960623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:25.960794 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.960776 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:25.960839 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.960796 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:25.960873 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:25.960844 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:33.960830319 +0000 UTC m=+67.382594782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:26.341901 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:26.341871 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:26.442400 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:26.442370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:27.122154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:27.122115 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:27.219523 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:27.219495 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:28.698283 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:28.698249 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:28.776956 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:28.776911 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:29.354408 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:29.354381 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-btvqp" Jul 03 14:09:29.587896 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:29.587864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:29.588071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:29.587910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:29.588071 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:29.587987 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 14:09:29.588071 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:29.588006 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:29.588071 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:29.588048 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:37.588033428 +0000 UTC m=+71.009797888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : configmap "kube-rbac-proxy" not found Jul 03 14:09:29.588071 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:29.588062 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:37.588056112 +0000 UTC m=+71.009820570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:29.703134 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:29.703058 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:32.812020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.811981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:32.815252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.815236 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 14:09:32.825092 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.825073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d02fb523-8f63-44bb-8b13-c7cb33393387-original-pull-secret\") pod \"global-pull-secret-syncer-xdtqp\" (UID: \"d02fb523-8f63-44bb-8b13-c7cb33393387\") " pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:32.912637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.912602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:32.912833 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.912650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:09:32.919260 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.919241 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 14:09:32.919409 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.919391 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 14:09:32.923563 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:32.923545 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 03 14:09:32.923649 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:32.923637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs podName:d07d0072-3662-42cf-86b5-8af045828769 nodeName:}" failed. No retries permitted until 2026-07-03 14:10:36.923621318 +0000 UTC m=+130.345385776 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs") pod "network-metrics-daemon-bc6rt" (UID: "d07d0072-3662-42cf-86b5-8af045828769") : secret "metrics-daemon-secret" not found Jul 03 14:09:32.926808 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.926795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xdtqp" Jul 03 14:09:32.933056 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.933018 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 14:09:32.936544 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:32.936518 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjbmx\" (UniqueName: \"kubernetes.io/projected/27d773c8-a973-4519-b705-dcae305e2294-kube-api-access-fjbmx\") pod \"network-check-target-mvptm\" (UID: \"27d773c8-a973-4519-b705-dcae305e2294\") " pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:33.075671 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:33.075640 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02fb523_8f63_44bb_8b13_c7cb33393387.slice/crio-19c8dc7a0ca3074f750fd8b0bba6bdc205ef8dfcb23df7a499af8328476439ae WatchSource:0}: Error finding container 19c8dc7a0ca3074f750fd8b0bba6bdc205ef8dfcb23df7a499af8328476439ae: Status 404 returned error can't find the container with id 19c8dc7a0ca3074f750fd8b0bba6bdc205ef8dfcb23df7a499af8328476439ae Jul 03 14:09:33.076776 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.076752 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xdtqp"] Jul 03 14:09:33.079447 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.079417 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.198784 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.198754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.224146 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.224126 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlbvf\"" Jul 03 14:09:33.232460 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.232442 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:33.309461 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.309425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.368025 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.367951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mvptm"] Jul 03 14:09:33.374356 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:33.374330 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d773c8_a973_4519_b705_dcae305e2294.slice/crio-8e43a6079cee1a3970a5c6c239ae01ecd74dcdb0cc968484c96cbdbc641cfc9b WatchSource:0}: Error finding container 8e43a6079cee1a3970a5c6c239ae01ecd74dcdb0cc968484c96cbdbc641cfc9b: Status 404 returned error can't find the container with id 8e43a6079cee1a3970a5c6c239ae01ecd74dcdb0cc968484c96cbdbc641cfc9b Jul 03 14:09:33.448159 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.448129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xdtqp" event={"ID":"d02fb523-8f63-44bb-8b13-c7cb33393387","Type":"ContainerStarted","Data":"19c8dc7a0ca3074f750fd8b0bba6bdc205ef8dfcb23df7a499af8328476439ae"} Jul 03 14:09:33.449118 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.449098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mvptm" event={"ID":"27d773c8-a973-4519-b705-dcae305e2294","Type":"ContainerStarted","Data":"8e43a6079cee1a3970a5c6c239ae01ecd74dcdb0cc968484c96cbdbc641cfc9b"} Jul 03 14:09:33.482989 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.482966 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.581650 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.581623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.746031 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.745939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:33.819441 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:33.819404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:09:33.819849 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:33.819560 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 14:09:33.819849 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:33.819634 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls podName:d89c1a98-bdb5-4a93-972b-dce4073fbf34 nodeName:}" failed. No retries permitted until 2026-07-03 14:10:05.819614248 +0000 UTC m=+99.241378728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls") pod "dns-default-j74ld" (UID: "d89c1a98-bdb5-4a93-972b-dce4073fbf34") : secret "dns-default-metrics-tls" not found Jul 03 14:09:34.021364 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:34.021281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:34.021529 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:34.021459 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 14:09:34.021529 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:34.021483 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64f84bdd7f-wrjcp: secret "image-registry-tls" not found Jul 03 14:09:34.021630 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:34.021551 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls podName:4d5e536a-a856-4bd9-9eac-b54b0982bb29 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:50.021535129 +0000 UTC m=+83.443299587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls") pod "image-registry-64f84bdd7f-wrjcp" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29") : secret "image-registry-tls" not found Jul 03 14:09:34.375539 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:34.375501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:34.522868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:34.522821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:34.939395 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:34.939356 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:35.041416 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:35.041379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:35.140334 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:35.140303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:35.508060 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:35.508025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:36.314332 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:36.314297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:36.691828 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:36.691792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:36.791626 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:36.791537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:36.926876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:36.926840 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:37.129208 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.129178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:37.539970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.539877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:37.639580 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.639548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:37.653616 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.653592 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:37.653795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.653630 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:37.653795 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:37.653773 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 14:09:37.653921 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:09:37.653839 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls podName:462292be-e44f-4b52-9608-0cb0207499f9 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:53.653820011 +0000 UTC m=+87.075584473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4s4gl" (UID: "462292be-e44f-4b52-9608-0cb0207499f9") : secret "insights-runtime-extractor-tls" not found Jul 03 14:09:37.654296 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:37.654263 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/462292be-e44f-4b52-9608-0cb0207499f9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:38.310154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.310121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:38.466125 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.466090 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xdtqp" event={"ID":"d02fb523-8f63-44bb-8b13-c7cb33393387","Type":"ContainerStarted","Data":"fe358fb2eacc62bc0e8cb9adab6ec1e5f5ae9e56e664434622124a8e8d0cfa57"} Jul 03 14:09:38.467667 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.467635 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mvptm" event={"ID":"27d773c8-a973-4519-b705-dcae305e2294","Type":"ContainerStarted","Data":"39dced14dfec053a89d5e50566d7047eaa8f8d85279e38ec9db9defde5a47425"} Jul 03 14:09:38.467812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.467784 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:09:38.492515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.492464 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xdtqp" podStartSLOduration=65.283398078 podStartE2EDuration="1m10.49244952s" totalImagesPullingTime="5.209051442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:38.492202931 +0000 UTC m=+71.913967412" watchObservedRunningTime="2026-07-03 14:09:38.49244952 +0000 UTC m=+71.914214001" Jul 03 14:09:38.513082 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.512974 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mvptm" podStartSLOduration=65.615805025 podStartE2EDuration="1m10.512954157s" totalImagesPullingTime="4.897149132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:38.512710467 +0000 UTC m=+71.934474951" watchObservedRunningTime="2026-07-03 14:09:38.512954157 +0000 UTC m=+71.934718639" Jul 03 14:09:38.737974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.737940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:38.917303 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:38.917276 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:39.092328 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:39.092293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:39.326993 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:39.326964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:39.426998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:39.426967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:40.161409 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:40.161371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:40.514629 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:40.514558 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:40.774093 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:40.774016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:41.339170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:41.339136 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:41.530316 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:41.530278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:41.724654 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:41.724575 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:42.314289 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:42.314256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:42.780300 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:42.780265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:43.228807 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:43.228778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:43.339815 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:43.339785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:43.622717 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:43.622687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:43.734817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:43.734787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:43.923300 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:43.923227 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.053605 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.053574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.220934 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.220863 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.401580 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.401551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.552677 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.552604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.649304 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.649279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:44.968173 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:44.968143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:45.391591 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:45.391555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:45.470790 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:45.470759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:45.573474 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:45.573448 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:45.783000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:45.782909 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:45.892079 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:45.892051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:47.590603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:47.590568 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:47.691862 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:47.691830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:50.053293 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.053150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:50.055507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.055481 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"image-registry-64f84bdd7f-wrjcp\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:50.268859 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.268829 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-scvmw\"" Jul 03 14:09:50.276984 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.276960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:50.405878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.405849 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:09:50.408612 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:50.408584 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5e536a_a856_4bd9_9eac_b54b0982bb29.slice/crio-11cbfd06313bbee1dd251d4bff28763aa01f59a752289c6ed45f669c19a5e31b WatchSource:0}: Error finding container 11cbfd06313bbee1dd251d4bff28763aa01f59a752289c6ed45f669c19a5e31b: Status 404 returned error can't find the container with id 11cbfd06313bbee1dd251d4bff28763aa01f59a752289c6ed45f669c19a5e31b Jul 03 14:09:50.502952 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.502924 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" event={"ID":"4d5e536a-a856-4bd9-9eac-b54b0982bb29","Type":"ContainerStarted","Data":"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4"} Jul 03 14:09:50.503079 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.502968 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" event={"ID":"4d5e536a-a856-4bd9-9eac-b54b0982bb29","Type":"ContainerStarted","Data":"11cbfd06313bbee1dd251d4bff28763aa01f59a752289c6ed45f669c19a5e31b"} Jul 03 14:09:50.503079 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.503036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:09:50.529440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:50.529395 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" podStartSLOduration=32.529380998 podStartE2EDuration="32.529380998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:50.529206892 +0000 UTC m=+83.950971374" watchObservedRunningTime="2026-07-03 14:09:50.529380998 +0000 UTC m=+83.951145457" Jul 03 14:09:51.634679 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.634649 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt"] Jul 03 14:09:51.639884 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.639865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.656498 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.656479 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 03 14:09:51.656703 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.656684 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 03 14:09:51.656790 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.656686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 03 14:09:51.657459 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.657445 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 03 14:09:51.657583 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.657567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 03 14:09:51.657662 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.657621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 03 14:09:51.657662 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.657629 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 03 14:09:51.690927 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.690903 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:09:51.702126 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.702103 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt"] Jul 03 14:09:51.769170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9968439-87c3-4140-aae5-b8dff012d9b6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.769170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.769351 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.769351 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.769351 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7k5\" (UniqueName: \"kubernetes.io/projected/a9968439-87c3-4140-aae5-b8dff012d9b6-kube-api-access-mt7k5\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.769351 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.769321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.869998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.869964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.869998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7k5\" (UniqueName: \"kubernetes.io/projected/a9968439-87c3-4140-aae5-b8dff012d9b6-kube-api-access-mt7k5\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.870182 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.870182 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9968439-87c3-4140-aae5-b8dff012d9b6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.870182 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.870336 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.871001 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.870975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9968439-87c3-4140-aae5-b8dff012d9b6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.872426 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.872403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-ca\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.872584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.872563 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.872663 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.872650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.872744 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.872706 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9968439-87c3-4140-aae5-b8dff012d9b6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.896930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.896880 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7k5\" (UniqueName: \"kubernetes.io/projected/a9968439-87c3-4140-aae5-b8dff012d9b6-kube-api-access-mt7k5\") pod \"cluster-proxy-proxy-agent-cb89dc47f-cvwxt\" (UID: \"a9968439-87c3-4140-aae5-b8dff012d9b6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:51.911474 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.911450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:51.958684 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:51.958656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" Jul 03 14:09:52.054661 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:52.054632 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:52.094472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:52.094447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt"] Jul 03 14:09:52.097898 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:52.097868 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9968439_87c3_4140_aae5_b8dff012d9b6.slice/crio-8a793f9a3d41fe8b5febf136b471f02a6d1499e468949bae58c61b39e9af628d WatchSource:0}: Error finding container 8a793f9a3d41fe8b5febf136b471f02a6d1499e468949bae58c61b39e9af628d: Status 404 returned error can't find the container with id 8a793f9a3d41fe8b5febf136b471f02a6d1499e468949bae58c61b39e9af628d Jul 03 14:09:52.244581 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:52.244504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:52.510808 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:52.510711 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" event={"ID":"a9968439-87c3-4140-aae5-b8dff012d9b6","Type":"ContainerStarted","Data":"8a793f9a3d41fe8b5febf136b471f02a6d1499e468949bae58c61b39e9af628d"} Jul 03 14:09:53.020695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.020660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:53.113827 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.113794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:53.206914 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.206885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:53.357181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.357140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:53.684536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.684385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:53.687240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.687210 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/462292be-e44f-4b52-9608-0cb0207499f9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4s4gl\" (UID: \"462292be-e44f-4b52-9608-0cb0207499f9\") " pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:53.987703 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.987637 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kwmd4\"" Jul 03 14:09:53.995818 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:53.995788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4s4gl" Jul 03 14:09:54.861825 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:54.861795 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4s4gl"] Jul 03 14:09:54.865825 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:09:54.865793 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462292be_e44f_4b52_9608_0cb0207499f9.slice/crio-ffbe37c52f947e4fae3383a41eadbbd53114659094f034437c92a20307bddfe9 WatchSource:0}: Error finding container ffbe37c52f947e4fae3383a41eadbbd53114659094f034437c92a20307bddfe9: Status 404 returned error can't find the container with id ffbe37c52f947e4fae3383a41eadbbd53114659094f034437c92a20307bddfe9 Jul 03 14:09:55.526196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:55.526160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4s4gl" event={"ID":"462292be-e44f-4b52-9608-0cb0207499f9","Type":"ContainerStarted","Data":"ffbe37c52f947e4fae3383a41eadbbd53114659094f034437c92a20307bddfe9"} Jul 03 14:09:56.533655 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.533569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" event={"ID":"a9968439-87c3-4140-aae5-b8dff012d9b6","Type":"ContainerStarted","Data":"86a0ef6855e4871299fc8b4a6d8b8b6a137325eb43e7099da3af0b6043a03f1f"} Jul 03 14:09:56.533655 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.533632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" event={"ID":"a9968439-87c3-4140-aae5-b8dff012d9b6","Type":"ContainerStarted","Data":"d4337e74cba54be4dfcdd08e6d1d9bf1edd22b32a6f73b36ac0d566bb5fedb07"} Jul 03 14:09:56.533655 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.533654 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" event={"ID":"a9968439-87c3-4140-aae5-b8dff012d9b6","Type":"ContainerStarted","Data":"39fc2517aa6cbfd3a35f57851d3a984ba2738f12921ae3cecfb81d552ac9c8ce"} Jul 03 14:09:56.558497 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.557276 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb89dc47f-cvwxt" podStartSLOduration=1.477915629 podStartE2EDuration="5.5572551s" totalImagesPullingTime="4.079339471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:56.556105105 +0000 UTC m=+89.977869587" watchObservedRunningTime="2026-07-03 14:09:56.5572551 +0000 UTC m=+89.979019582" Jul 03 14:09:56.600327 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.600292 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:56.700427 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:56.700390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:58.392888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:58.392844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:09:58.466482 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:09:58.466452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:00.484102 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:00.484025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:00.548470 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:00.548440 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4s4gl" event={"ID":"462292be-e44f-4b52-9608-0cb0207499f9","Type":"ContainerStarted","Data":"fba6fc3962ac1eb24196f65850ccdb87f3b9e4399f4870ffc74fc170b90f91d9"} Jul 03 14:10:00.548610 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:00.548491 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4s4gl" event={"ID":"462292be-e44f-4b52-9608-0cb0207499f9","Type":"ContainerStarted","Data":"a5aba28caca6725660909cc2863c9c9e49d11e2b372153a619ee3a1f38fa442c"} Jul 03 14:10:00.637415 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:00.637383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:01.256307 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:01.256266 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:01.390181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:01.390143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:01.532789 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:01.532685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:03.387908 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:03.387863 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:03.452743 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:03.452699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:03.560639 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:03.560604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4s4gl" event={"ID":"462292be-e44f-4b52-9608-0cb0207499f9","Type":"ContainerStarted","Data":"3967a1a9a859b7356f9f6e91ff367264f3998557e330ab52f21cdbfe4a7ad7ec"} Jul 03 14:10:03.581076 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:03.581028 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4s4gl" podStartSLOduration=34.904861479 podStartE2EDuration="42.581012408s" totalImagesPullingTime="7.676150929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:09:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:03.580468063 +0000 UTC m=+97.002232545" watchObservedRunningTime="2026-07-03 14:10:03.581012408 +0000 UTC m=+97.002776889" Jul 03 14:10:04.754894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:04.754858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:05.040027 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:05.039939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:05.113219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:05.113192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:05.894385 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:05.894349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:10:05.896580 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:05.896559 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d89c1a98-bdb5-4a93-972b-dce4073fbf34-metrics-tls\") pod \"dns-default-j74ld\" (UID: \"d89c1a98-bdb5-4a93-972b-dce4073fbf34\") " pod="openshift-dns/dns-default-j74ld" Jul 03 14:10:06.127393 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.127364 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wf6pg\"" Jul 03 14:10:06.134876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.134840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j74ld" Jul 03 14:10:06.266144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.266113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j74ld"] Jul 03 14:10:06.268721 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:10:06.268692 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89c1a98_bdb5_4a93_972b_dce4073fbf34.slice/crio-ae608a00142b816f19f73d7a25b978053e3149442f1840d86c7e41594a71e346 WatchSource:0}: Error finding container ae608a00142b816f19f73d7a25b978053e3149442f1840d86c7e41594a71e346: Status 404 returned error can't find the container with id ae608a00142b816f19f73d7a25b978053e3149442f1840d86c7e41594a71e346 Jul 03 14:10:06.442426 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.442351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:06.570380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.570341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j74ld" event={"ID":"d89c1a98-bdb5-4a93-972b-dce4073fbf34","Type":"ContainerStarted","Data":"ae608a00142b816f19f73d7a25b978053e3149442f1840d86c7e41594a71e346"} Jul 03 14:10:06.648307 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.648269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:06.743125 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.743046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:06.779335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.779283 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mtcxj"] Jul 03 14:10:06.792587 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.792551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.795202 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.795170 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 14:10:06.795329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.795209 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 14:10:06.795329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.795170 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 14:10:06.796360 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.796331 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-54cgl\"" Jul 03 14:10:06.796501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.796370 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 14:10:06.796562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.796533 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 14:10:06.796788 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.796770 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 14:10:06.858127 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.858083 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:06.901787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901750 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-wtmp\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.901787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-root\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-textfile\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901846 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-metrics-client-ca\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-sys\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.901995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9k8m\" (UniqueName: \"kubernetes.io/projected/3238b6ed-5d74-4933-a005-ebf6eeab8eca-kube-api-access-v9k8m\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.902261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.902018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:06.970526 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:06.970491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.002596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-sys\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9k8m\" (UniqueName: \"kubernetes.io/projected/3238b6ed-5d74-4933-a005-ebf6eeab8eca-kube-api-access-v9k8m\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-wtmp\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002648 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-sys\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:10:07.002685 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-root\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-root\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:10:07.002780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls podName:3238b6ed-5d74-4933-a005-ebf6eeab8eca nodeName:}" failed. No retries permitted until 2026-07-03 14:10:07.502758171 +0000 UTC m=+100.924522643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls") pod "node-exporter-mtcxj" (UID: "3238b6ed-5d74-4933-a005-ebf6eeab8eca") : secret "node-exporter-tls" not found Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002839 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-wtmp\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.002937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-textfile\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.003370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.003370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.002997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-metrics-client-ca\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.003370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.003308 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-textfile\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.003695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.003675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.004022 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.003996 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3238b6ed-5d74-4933-a005-ebf6eeab8eca-metrics-client-ca\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.005379 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.005354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.012004 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.011976 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9k8m\" (UniqueName: \"kubernetes.io/projected/3238b6ed-5d74-4933-a005-ebf6eeab8eca-kube-api-access-v9k8m\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.071551 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.071514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.172157 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.172124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.506517 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.506478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:07.506691 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:10:07.506647 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 14:10:07.506768 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:10:07.506720 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls podName:3238b6ed-5d74-4933-a005-ebf6eeab8eca nodeName:}" failed. No retries permitted until 2026-07-03 14:10:08.506703848 +0000 UTC m=+101.928468310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls") pod "node-exporter-mtcxj" (UID: "3238b6ed-5d74-4933-a005-ebf6eeab8eca") : secret "node-exporter-tls" not found Jul 03 14:10:07.572935 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.572901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.675017 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.674972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.778280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.778198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.880660 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.880624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:07.998561 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:07.998528 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:08.515191 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:08.515160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:08.517371 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:08.517349 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3238b6ed-5d74-4933-a005-ebf6eeab8eca-node-exporter-tls\") pod \"node-exporter-mtcxj\" (UID: \"3238b6ed-5d74-4933-a005-ebf6eeab8eca\") " pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:08.578133 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:08.578101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j74ld" event={"ID":"d89c1a98-bdb5-4a93-972b-dce4073fbf34","Type":"ContainerStarted","Data":"33aa5ae280b4916bc76fac5341314ea22ae80f610f3d738c29aff43d2d441d13"} Jul 03 14:10:08.607883 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:08.607836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mtcxj" Jul 03 14:10:08.624400 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:10:08.624380 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3238b6ed_5d74_4933_a005_ebf6eeab8eca.slice/crio-243675ca25035758227cc87c6a6c456dc8ce354833c8578fe1c51e4b6488519e WatchSource:0}: Error finding container 243675ca25035758227cc87c6a6c456dc8ce354833c8578fe1c51e4b6488519e: Status 404 returned error can't find the container with id 243675ca25035758227cc87c6a6c456dc8ce354833c8578fe1c51e4b6488519e Jul 03 14:10:09.473017 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.472985 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mvptm" Jul 03 14:10:09.585482 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.585405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j74ld" event={"ID":"d89c1a98-bdb5-4a93-972b-dce4073fbf34","Type":"ContainerStarted","Data":"975f454139558ccf9a79051e399d48c13e854934451ce1a556f4886526134603"} Jul 03 14:10:09.585656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.585546 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j74ld" Jul 03 14:10:09.586836 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.586813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtcxj" event={"ID":"3238b6ed-5d74-4933-a005-ebf6eeab8eca","Type":"ContainerStarted","Data":"243675ca25035758227cc87c6a6c456dc8ce354833c8578fe1c51e4b6488519e"} Jul 03 14:10:09.613341 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.613286 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j74ld" podStartSLOduration=66.527526453 podStartE2EDuration="1m8.613268133s" totalImagesPullingTime="2.08574168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:09.612770938 +0000 UTC m=+103.034535420" watchObservedRunningTime="2026-07-03 14:10:09.613268133 +0000 UTC m=+103.035032621" Jul 03 14:10:09.806420 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.806320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:09.921831 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:09.921799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:10.065323 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:10.065263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:10.260331 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:10.260292 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:10.375989 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:10.375958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:10.591169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:10.591132 2580 generic.go:356] "Generic (PLEG): container finished" podID="3238b6ed-5d74-4933-a005-ebf6eeab8eca" containerID="05ed2cd22bea668f7467c38d4d9a380de349cb2ad367ac1b76940008bb79646b" exitCode=0 Jul 03 14:10:10.591565 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:10.591216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtcxj" event={"ID":"3238b6ed-5d74-4933-a005-ebf6eeab8eca","Type":"ContainerDied","Data":"05ed2cd22bea668f7467c38d4d9a380de349cb2ad367ac1b76940008bb79646b"} Jul 03 14:10:11.433499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.433454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:11.596608 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.596568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtcxj" event={"ID":"3238b6ed-5d74-4933-a005-ebf6eeab8eca","Type":"ContainerStarted","Data":"60fce2f1e626b8663509d7df554e9ea68fd4f4a6c495f20bad401006cc8d737a"} Jul 03 14:10:11.596998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.596620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtcxj" event={"ID":"3238b6ed-5d74-4933-a005-ebf6eeab8eca","Type":"ContainerStarted","Data":"ace54c570ddd3ccd44dae5c76e63c52df7509dc881527b4f13a87b36320702be"} Jul 03 14:10:11.609282 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.609255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:11.633002 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.632954 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mtcxj" podStartSLOduration=4.131939609 podStartE2EDuration="5.632937103s" totalImagesPullingTime="1.39008105s" totalInitContainerRuntime="110.916444ms" isStatefulPod=true podCreationTimestamp="2026-07-03 14:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:11.632248494 +0000 UTC m=+105.054012975" watchObservedRunningTime="2026-07-03 14:10:11.632937103 +0000 UTC m=+105.054701584" Jul 03 14:10:11.876154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:11.876119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:12.422832 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:12.422802 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:12.516499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:12.516471 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:10:12.745770 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:12.745673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:13.419828 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.419796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:13.697341 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.697252 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:10:13.699279 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.699257 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.706533 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.703643 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 14:10:13.706533 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.703958 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 14:10:13.706533 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.704192 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 14:10:13.709631 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.709597 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-svd8d\"" Jul 03 14:10:13.709753 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.709702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 14:10:13.710225 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.710209 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 14:10:13.710477 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.710457 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 14:10:13.710584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.710560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 14:10:13.740242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.740217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 14:10:13.754538 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754582 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4r8\" (UniqueName: \"kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.754878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.754714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.777821 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.777803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:10:13.844043 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.843997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:13.855468 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4r8\" (UniqueName: \"kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855762 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855578 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.855762 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.855600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.856350 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.856317 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.856452 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.856375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.856452 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.856428 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.856529 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.856499 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.858030 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.858003 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.858177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.858159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:13.900901 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:13.900871 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4r8\" (UniqueName: \"kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8\") pod \"console-5b9b5869c6-f5mn8\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:14.011372 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.011276 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:14.102851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.102823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:14.156230 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.156204 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:10:14.158435 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:10:14.158414 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e78c50_fc3d_439d_915d_9d535d9ee29d.slice/crio-707208c9f49e41b155956aeaddac5b67dd07d857d8fa17e4b502fcdb37fd6db4 WatchSource:0}: Error finding container 707208c9f49e41b155956aeaddac5b67dd07d857d8fa17e4b502fcdb37fd6db4: Status 404 returned error can't find the container with id 707208c9f49e41b155956aeaddac5b67dd07d857d8fa17e4b502fcdb37fd6db4 Jul 03 14:10:14.607692 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.607658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9b5869c6-f5mn8" event={"ID":"36e78c50-fc3d-439d-915d-9d535d9ee29d","Type":"ContainerStarted","Data":"707208c9f49e41b155956aeaddac5b67dd07d857d8fa17e4b502fcdb37fd6db4"} Jul 03 14:10:14.637561 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.637531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:14.976338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:14.976261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:15.153254 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:15.153225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:16.637586 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:16.637547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:16.709010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:16.708980 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:17.529642 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.529542 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" podUID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" containerName="registry" containerID="cri-o://69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4" gracePeriod=30 Jul 03 14:10:17.619440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.619407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9b5869c6-f5mn8" event={"ID":"36e78c50-fc3d-439d-915d-9d535d9ee29d","Type":"ContainerStarted","Data":"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc"} Jul 03 14:10:17.758576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.758552 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:10:17.779965 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.779880 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b9b5869c6-f5mn8" podStartSLOduration=1.849010043 podStartE2EDuration="4.779865442s" totalImagesPullingTime="2.930855399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:17.65203609 +0000 UTC m=+111.073800550" watchObservedRunningTime="2026-07-03 14:10:17.779865442 +0000 UTC m=+111.201629923" Jul 03 14:10:17.785455 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785463 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlgdn\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785484 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785513 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785550 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785581 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785615 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.785960 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.785935 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted\") pod \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\" (UID: \"4d5e536a-a856-4bd9-9eac-b54b0982bb29\") " Jul 03 14:10:17.786408 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.786288 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:10:17.786553 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.786455 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:10:17.789644 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.789615 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn" (OuterVolumeSpecName: "kube-api-access-tlgdn") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "kube-api-access-tlgdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:10:17.789855 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.789807 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:10:17.789934 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.789890 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:10:17.790001 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.789985 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:10:17.790281 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.790259 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:10:17.795380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.795357 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4d5e536a-a856-4bd9-9eac-b54b0982bb29" (UID: "4d5e536a-a856-4bd9-9eac-b54b0982bb29"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:10:17.887515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887473 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-trusted-ca\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887511 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-bound-sa-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887520 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4d5e536a-a856-4bd9-9eac-b54b0982bb29-ca-trust-extracted\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887515 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887530 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-installation-pull-secrets\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887538 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlgdn\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-kube-api-access-tlgdn\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887547 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-tls\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887555 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4d5e536a-a856-4bd9-9eac-b54b0982bb29-registry-certificates\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:17.887834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:17.887564 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4d5e536a-a856-4bd9-9eac-b54b0982bb29-image-registry-private-configuration\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:10:18.624135 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.624101 2580 generic.go:356] "Generic (PLEG): container finished" podID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" containerID="69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4" exitCode=0 Jul 03 14:10:18.624299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.624146 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" Jul 03 14:10:18.624299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.624186 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" event={"ID":"4d5e536a-a856-4bd9-9eac-b54b0982bb29","Type":"ContainerDied","Data":"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4"} Jul 03 14:10:18.624299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.624235 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64f84bdd7f-wrjcp" event={"ID":"4d5e536a-a856-4bd9-9eac-b54b0982bb29","Type":"ContainerDied","Data":"11cbfd06313bbee1dd251d4bff28763aa01f59a752289c6ed45f669c19a5e31b"} Jul 03 14:10:18.624299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.624261 2580 scope.go:117] "RemoveContainer" containerID="69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4" Jul 03 14:10:18.633597 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.633578 2580 scope.go:117] "RemoveContainer" containerID="69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4" Jul 03 14:10:18.633845 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:10:18.633821 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4\": container with ID starting with 69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4 not found: ID does not exist" containerID="69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4" Jul 03 14:10:18.633894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.633855 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4"} err="failed to get container status \"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4\": rpc error: code = NotFound desc = could not find container \"69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4\": container with ID starting with 69b2b55667ca0d4152c328c364049036af0eacf6239de92f6b077a33086343e4 not found: ID does not exist" Jul 03 14:10:18.653795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.653768 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:10:18.659858 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:18.659840 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64f84bdd7f-wrjcp"] Jul 03 14:10:19.203876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:19.203844 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" path="/var/lib/kubelet/pods/4d5e536a-a856-4bd9-9eac-b54b0982bb29/volumes" Jul 03 14:10:19.593460 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:19.593429 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j74ld" Jul 03 14:10:20.142405 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:20.142377 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:20.245509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:20.245482 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:20.795059 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:20.795028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:20.899455 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:20.899426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:21.310961 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:21.310916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:21.428435 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:21.428404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:24.012375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:24.012340 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:24.012940 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:24.012397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:24.017607 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:24.017583 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:24.648620 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:24.648593 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:10:25.160004 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:25.159973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:25.323199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:25.323163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:28.878122 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:28.878089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:29.023322 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:29.023289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:29.649005 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:29.648979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:29.753137 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:29.753092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:30.761377 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:30.761352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:30.931356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:30.931327 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:31.026963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:31.026890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:35.064374 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:35.064342 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:35.167386 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:35.167351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:36.925522 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:36.925476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:10:36.927867 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:36.927845 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d07d0072-3662-42cf-86b5-8af045828769-metrics-certs\") pod \"network-metrics-daemon-bc6rt\" (UID: \"d07d0072-3662-42cf-86b5-8af045828769\") " pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:10:37.117673 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:37.117643 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-472cc\"" Jul 03 14:10:37.125516 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:37.125487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bc6rt" Jul 03 14:10:37.293428 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:37.293399 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bc6rt"] Jul 03 14:10:37.296760 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:10:37.296720 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07d0072_3662_42cf_86b5_8af045828769.slice/crio-3704b228eda26d0e33b45aa7a2488cbdaf1161d4664dc902ddd3c87832ed7284 WatchSource:0}: Error finding container 3704b228eda26d0e33b45aa7a2488cbdaf1161d4664dc902ddd3c87832ed7284: Status 404 returned error can't find the container with id 3704b228eda26d0e33b45aa7a2488cbdaf1161d4664dc902ddd3c87832ed7284 Jul 03 14:10:37.687243 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:37.687208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bc6rt" event={"ID":"d07d0072-3662-42cf-86b5-8af045828769","Type":"ContainerStarted","Data":"3704b228eda26d0e33b45aa7a2488cbdaf1161d4664dc902ddd3c87832ed7284"} Jul 03 14:10:38.692685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:38.692594 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bc6rt" event={"ID":"d07d0072-3662-42cf-86b5-8af045828769","Type":"ContainerStarted","Data":"526d6bdb21184e6f31fde2eb3946ea680e4b1b5b699f7d5ff65a0eef82e99cf9"} Jul 03 14:10:38.692685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:38.692650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bc6rt" event={"ID":"d07d0072-3662-42cf-86b5-8af045828769","Type":"ContainerStarted","Data":"09a753efcbc86a748ef7f23471442d5aa2ad85348969e607e88671d06334f02b"} Jul 03 14:10:38.883774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:38.883744 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:38.969527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:38.969449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:46.719926 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:46.719839 2580 generic.go:356] "Generic (PLEG): container finished" podID="bde77271-76bc-428a-a4c4-f096114b6526" containerID="2ad443387b29fa5b699ce58280a1c712e2d822a568e474424f196955c5af6672" exitCode=0 Jul 03 14:10:46.719926 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:46.719876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" event={"ID":"bde77271-76bc-428a-a4c4-f096114b6526","Type":"ContainerDied","Data":"2ad443387b29fa5b699ce58280a1c712e2d822a568e474424f196955c5af6672"} Jul 03 14:10:46.720335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:46.720175 2580 scope.go:117] "RemoveContainer" containerID="2ad443387b29fa5b699ce58280a1c712e2d822a568e474424f196955c5af6672" Jul 03 14:10:46.745036 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:46.744989 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bc6rt" podStartSLOduration=137.701054042 podStartE2EDuration="2m18.744975962s" totalImagesPullingTime="1.04392192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:38.718514918 +0000 UTC m=+132.140279401" watchObservedRunningTime="2026-07-03 14:10:46.744975962 +0000 UTC m=+140.166740471" Jul 03 14:10:47.724851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:47.724817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ccctb" event={"ID":"bde77271-76bc-428a-a4c4-f096114b6526","Type":"ContainerStarted","Data":"c0f2b6feea02eb3e84d1d8a6564a0ad74cccc6aa2ae270088101b2a904f25b44"} Jul 03 14:10:50.039092 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:50.039056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:10:50.139891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:10:50.139858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:00.790215 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:00.790181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:04.611586 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:04.611557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:22.852023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:22.851943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:26.231539 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:26.231507 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:26.470240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:26.470206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:35.851630 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.851596 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:11:35.852047 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.851842 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" containerName="registry" Jul 03 14:11:35.852047 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.851854 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" containerName="registry" Jul 03 14:11:35.852047 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.851891 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d5e536a-a856-4bd9-9eac-b54b0982bb29" containerName="registry" Jul 03 14:11:35.853437 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.853421 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.871335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.871311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:11:35.932116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932398 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbr4d\" (UniqueName: \"kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.932398 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.932339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:35.944787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:35.944759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:36.033333 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033333 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033559 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033559 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbr4d\" (UniqueName: \"kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033559 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033710 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.033710 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.033623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.034188 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.034142 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.034297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.034152 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.034297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.034289 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.034392 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.034334 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.035942 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.035923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.036356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.036339 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.043473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.043449 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbr4d\" (UniqueName: \"kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d\") pod \"console-5f59b4bfdc-zsrct\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.053590 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.053568 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:36.162079 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.161993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:36.290054 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.289873 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:11:36.292735 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:11:36.292696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b10aaa_fda9_4ac2_8786_2b559e6570e8.slice/crio-9c8ff7fa59eea7653b093774c38aae39e257111b1ca788b924def1a1493b269f WatchSource:0}: Error finding container 9c8ff7fa59eea7653b093774c38aae39e257111b1ca788b924def1a1493b269f: Status 404 returned error can't find the container with id 9c8ff7fa59eea7653b093774c38aae39e257111b1ca788b924def1a1493b269f Jul 03 14:11:36.892480 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.892444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59b4bfdc-zsrct" event={"ID":"51b10aaa-fda9-4ac2-8786-2b559e6570e8","Type":"ContainerStarted","Data":"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999"} Jul 03 14:11:36.892878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.892499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59b4bfdc-zsrct" event={"ID":"51b10aaa-fda9-4ac2-8786-2b559e6570e8","Type":"ContainerStarted","Data":"9c8ff7fa59eea7653b093774c38aae39e257111b1ca788b924def1a1493b269f"} Jul 03 14:11:36.922892 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:36.922840 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f59b4bfdc-zsrct" podStartSLOduration=1.92282399 podStartE2EDuration="1.92282399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:11:36.921835455 +0000 UTC m=+190.343599936" watchObservedRunningTime="2026-07-03 14:11:36.92282399 +0000 UTC m=+190.344588470" Jul 03 14:11:46.162575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:46.162538 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:46.163099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:46.162830 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:46.167341 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:46.167324 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:46.928879 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:46.928846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:11:47.001640 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:47.001609 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:11:47.245244 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:47.245146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:47.358933 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:47.358908 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:11:47.442373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:11:47.442336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:12.020923 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.020869 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b9b5869c6-f5mn8" podUID="36e78c50-fc3d-439d-915d-9d535d9ee29d" containerName="console" containerID="cri-o://0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc" gracePeriod=15 Jul 03 14:12:12.249312 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.249287 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:12:12.293863 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.293796 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.293863 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.293830 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.293923 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.293964 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc4r8\" (UniqueName: \"kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.293998 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294055 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294089 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert\") pod \"36e78c50-fc3d-439d-915d-9d535d9ee29d\" (UID: \"36e78c50-fc3d-439d-915d-9d535d9ee29d\") " Jul 03 14:12:12.294252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294223 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config" (OuterVolumeSpecName: "console-config") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:12:12.294362 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294233 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:12:12.294362 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294311 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca" (OuterVolumeSpecName: "service-ca") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:12:12.294469 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.294411 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:12:12.296196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.296172 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:12:12.296196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.296177 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8" (OuterVolumeSpecName: "kube-api-access-zc4r8") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "kube-api-access-zc4r8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:12:12.296196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.296190 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36e78c50-fc3d-439d-915d-9d535d9ee29d" (UID: "36e78c50-fc3d-439d-915d-9d535d9ee29d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:12:12.395286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395242 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-service-ca\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395281 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc4r8\" (UniqueName: \"kubernetes.io/projected/36e78c50-fc3d-439d-915d-9d535d9ee29d-kube-api-access-zc4r8\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395294 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-oauth-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395308 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-trusted-ca-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395321 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395332 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-console-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:12.395504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:12.395342 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36e78c50-fc3d-439d-915d-9d535d9ee29d-oauth-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:12:13.013599 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.013563 2580 generic.go:356] "Generic (PLEG): container finished" podID="36e78c50-fc3d-439d-915d-9d535d9ee29d" containerID="0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc" exitCode=0 Jul 03 14:12:13.013809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.013609 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b9b5869c6-f5mn8" Jul 03 14:12:13.013809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.013623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9b5869c6-f5mn8" event={"ID":"36e78c50-fc3d-439d-915d-9d535d9ee29d","Type":"ContainerDied","Data":"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc"} Jul 03 14:12:13.013809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.013669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b9b5869c6-f5mn8" event={"ID":"36e78c50-fc3d-439d-915d-9d535d9ee29d","Type":"ContainerDied","Data":"707208c9f49e41b155956aeaddac5b67dd07d857d8fa17e4b502fcdb37fd6db4"} Jul 03 14:12:13.013809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.013695 2580 scope.go:117] "RemoveContainer" containerID="0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc" Jul 03 14:12:13.022451 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.022224 2580 scope.go:117] "RemoveContainer" containerID="0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc" Jul 03 14:12:13.022668 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:12:13.022481 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc\": container with ID starting with 0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc not found: ID does not exist" containerID="0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc" Jul 03 14:12:13.022668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.022511 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc"} err="failed to get container status \"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc\": rpc error: code = NotFound desc = could not find container \"0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc\": container with ID starting with 0a5dc22650ece63f36af3e3ffda9d137f9c8aee37edef4ec11040aaf991da6dc not found: ID does not exist" Jul 03 14:12:13.040625 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.040603 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:12:13.049107 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.049088 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b9b5869c6-f5mn8"] Jul 03 14:12:13.203877 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:13.203845 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e78c50-fc3d-439d-915d-9d535d9ee29d" path="/var/lib/kubelet/pods/36e78c50-fc3d-439d-915d-9d535d9ee29d/volumes" Jul 03 14:12:17.497940 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:17.497876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:17.603575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:17.603532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:41.696566 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.696531 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:12:41.697053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.696841 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e78c50-fc3d-439d-915d-9d535d9ee29d" containerName="console" Jul 03 14:12:41.697053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.696858 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e78c50-fc3d-439d-915d-9d535d9ee29d" containerName="console" Jul 03 14:12:41.697053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.696913 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e78c50-fc3d-439d-915d-9d535d9ee29d" containerName="console" Jul 03 14:12:41.699018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.699003 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.712612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.712581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:41.719527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.719503 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:12:41.792249 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792249 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dz9x\" (UniqueName: \"kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.792472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.792438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.825249 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.825220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:41.893120 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893120 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.893620 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dz9x\" (UniqueName: \"kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.894006 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.893984 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.894183 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.894074 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.894281 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.894177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.894281 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.894201 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.894664 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.894642 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:41.895695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.895675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.895868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.895851 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:41.902245 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:41.902225 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dz9x\" (UniqueName: \"kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x\") pod \"console-7c56bf885c-rjgn9\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:42.008453 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:42.008369 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:42.135035 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:42.134999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:12:42.139129 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:12:42.139100 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a38c82_6a27_43af_a4ed_92b9e9f7f316.slice/crio-3938c66a50787709f6751c66b9d7bad6363e9060e75478898f17cf6c5c189e17 WatchSource:0}: Error finding container 3938c66a50787709f6751c66b9d7bad6363e9060e75478898f17cf6c5c189e17: Status 404 returned error can't find the container with id 3938c66a50787709f6751c66b9d7bad6363e9060e75478898f17cf6c5c189e17 Jul 03 14:12:43.116604 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:43.116570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c56bf885c-rjgn9" event={"ID":"d3a38c82-6a27-43af-a4ed-92b9e9f7f316","Type":"ContainerStarted","Data":"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc"} Jul 03 14:12:43.116995 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:43.116617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c56bf885c-rjgn9" event={"ID":"d3a38c82-6a27-43af-a4ed-92b9e9f7f316","Type":"ContainerStarted","Data":"3938c66a50787709f6751c66b9d7bad6363e9060e75478898f17cf6c5c189e17"} Jul 03 14:12:43.142746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:43.142683 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c56bf885c-rjgn9" podStartSLOduration=2.142668224 podStartE2EDuration="2.142668224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:12:43.142407174 +0000 UTC m=+256.564171668" watchObservedRunningTime="2026-07-03 14:12:43.142668224 +0000 UTC m=+256.564432704" Jul 03 14:12:52.008848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.008758 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:52.008848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.008832 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:52.013508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.013485 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:52.149990 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.149957 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:12:52.204308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.204268 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:12:52.328394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.328364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:52.428615 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.428583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:52.493240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:52.493210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:53.592601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:53.592573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:12:53.696109 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:12:53.696078 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:13:05.069050 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:05.069008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:13:05.168964 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:05.168935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:13:17.228904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.228843 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f59b4bfdc-zsrct" podUID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" containerName="console" containerID="cri-o://2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999" gracePeriod=15 Jul 03 14:13:17.461101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.461079 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:13:17.553911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.553827 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.553911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.553868 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.553911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.553906 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbr4d\" (UniqueName: \"kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.554179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.554179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554069 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.554179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554094 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.554179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554142 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert\") pod \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\" (UID: \"51b10aaa-fda9-4ac2-8786-2b559e6570e8\") " Jul 03 14:13:17.554369 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554303 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:13:17.554420 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554373 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config" (OuterVolumeSpecName: "console-config") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:13:17.554603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554537 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:13:17.554763 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.554632 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca" (OuterVolumeSpecName: "service-ca") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:13:17.556154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.556127 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:13:17.556253 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.556147 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:13:17.556253 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.556129 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d" (OuterVolumeSpecName: "kube-api-access-wbr4d") pod "51b10aaa-fda9-4ac2-8786-2b559e6570e8" (UID: "51b10aaa-fda9-4ac2-8786-2b559e6570e8"). InnerVolumeSpecName "kube-api-access-wbr4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:13:17.655631 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655596 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655631 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655626 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-trusted-ca-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655638 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-service-ca\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655652 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655667 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51b10aaa-fda9-4ac2-8786-2b559e6570e8-oauth-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655679 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51b10aaa-fda9-4ac2-8786-2b559e6570e8-console-oauth-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:17.655849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:17.655690 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbr4d\" (UniqueName: \"kubernetes.io/projected/51b10aaa-fda9-4ac2-8786-2b559e6570e8-kube-api-access-wbr4d\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:13:18.238188 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.238156 2580 generic.go:356] "Generic (PLEG): container finished" podID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" containerID="2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999" exitCode=0 Jul 03 14:13:18.238588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.238204 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59b4bfdc-zsrct" Jul 03 14:13:18.238588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.238241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59b4bfdc-zsrct" event={"ID":"51b10aaa-fda9-4ac2-8786-2b559e6570e8","Type":"ContainerDied","Data":"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999"} Jul 03 14:13:18.238588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.238288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59b4bfdc-zsrct" event={"ID":"51b10aaa-fda9-4ac2-8786-2b559e6570e8","Type":"ContainerDied","Data":"9c8ff7fa59eea7653b093774c38aae39e257111b1ca788b924def1a1493b269f"} Jul 03 14:13:18.238588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.238309 2580 scope.go:117] "RemoveContainer" containerID="2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999" Jul 03 14:13:18.246620 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.246605 2580 scope.go:117] "RemoveContainer" containerID="2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999" Jul 03 14:13:18.246909 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:13:18.246890 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999\": container with ID starting with 2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999 not found: ID does not exist" containerID="2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999" Jul 03 14:13:18.247008 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.246917 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999"} err="failed to get container status \"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999\": rpc error: code = NotFound desc = could not find container \"2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999\": container with ID starting with 2b7af1f049de7d5ca5b9dbc876ea20f3ca11be2dcb66a324fb5b1befa0789999 not found: ID does not exist" Jul 03 14:13:18.262149 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.262129 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:13:18.266394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:18.266373 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f59b4bfdc-zsrct"] Jul 03 14:13:19.203924 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:19.203890 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" path="/var/lib/kubelet/pods/51b10aaa-fda9-4ac2-8786-2b559e6570e8/volumes" Jul 03 14:13:27.008589 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:27.008562 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 14:13:48.942461 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:48.942429 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:13:50.064102 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:13:50.064068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:36.286408 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:36.286334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:36.383041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:36.383012 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:36.566538 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:36.566455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:37.534024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:37.533997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:40.386824 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:40.386791 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:40.489543 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:40.489505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:42.137637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.137605 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:42.273562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.273527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:42.701766 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.701717 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m"] Jul 03 14:14:42.701980 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.701969 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" containerName="console" Jul 03 14:14:42.702027 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.701982 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" containerName="console" Jul 03 14:14:42.702027 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.702018 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b10aaa-fda9-4ac2-8786-2b559e6570e8" containerName="console" Jul 03 14:14:42.704938 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.704921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.707588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.707570 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 14:14:42.708428 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.708415 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 14:14:42.708487 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.708444 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d7mkw\"" Jul 03 14:14:42.725489 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.725464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m"] Jul 03 14:14:42.730366 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.730342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.730491 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.730372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ccpp\" (UniqueName: \"kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.730491 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.730397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.737741 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.737691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:42.828387 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.828352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:42.831404 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.831387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.831464 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.831412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ccpp\" (UniqueName: \"kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.831464 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.831438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.831793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.831775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.831836 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.831798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:42.840792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:42.840769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ccpp\" (UniqueName: \"kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:43.014044 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:43.013953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:14:43.153254 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:43.153224 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m"] Jul 03 14:14:43.156369 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:14:43.156344 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aeb6997_dad2_481c_8f8f_000359703cfe.slice/crio-f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449 WatchSource:0}: Error finding container f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449: Status 404 returned error can't find the container with id f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449 Jul 03 14:14:43.158268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:43.158249 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:14:43.513588 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:43.513559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerStarted","Data":"f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449"} Jul 03 14:14:44.907266 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:44.907228 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:45.117907 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.117874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:45.217949 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.217880 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-whbcm"] Jul 03 14:14:45.220764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.220740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.224330 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.224311 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 03 14:14:45.225180 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.225157 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mvkbq\"" Jul 03 14:14:45.225180 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.225176 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 03 14:14:45.245363 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.245329 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-whbcm"] Jul 03 14:14:45.252129 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.252100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc7mh\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-kube-api-access-vc7mh\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.252255 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.252146 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.353195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.353158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.353373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.353265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc7mh\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-kube-api-access-vc7mh\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.362576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.362547 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.362714 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.362589 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc7mh\" (UniqueName: \"kubernetes.io/projected/073d42fc-0af3-4b58-8006-2b4145773712-kube-api-access-vc7mh\") pod \"cert-manager-webhook-6486ffd997-whbcm\" (UID: \"073d42fc-0af3-4b58-8006-2b4145773712\") " pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.530574 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.530496 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:45.678175 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:45.678134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-whbcm"] Jul 03 14:14:45.681553 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:14:45.681526 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod073d42fc_0af3_4b58_8006_2b4145773712.slice/crio-92afc362c86f4c507c0ab3bbd7f35e88f0f1b7cdd1319feab01379bf7910f06b WatchSource:0}: Error finding container 92afc362c86f4c507c0ab3bbd7f35e88f0f1b7cdd1319feab01379bf7910f06b: Status 404 returned error can't find the container with id 92afc362c86f4c507c0ab3bbd7f35e88f0f1b7cdd1319feab01379bf7910f06b Jul 03 14:14:46.527479 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:46.527443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" event={"ID":"073d42fc-0af3-4b58-8006-2b4145773712","Type":"ContainerStarted","Data":"92afc362c86f4c507c0ab3bbd7f35e88f0f1b7cdd1319feab01379bf7910f06b"} Jul 03 14:14:47.202605 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:47.202517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:47.307793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:47.307760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:50.544005 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:50.543912 2580 generic.go:356] "Generic (PLEG): container finished" podID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerID="43573dd3ad95b159744c3552e2172f5cc9b9d1e790ae50bf02183e804ebbfb6f" exitCode=0 Jul 03 14:14:50.544454 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:50.544010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerDied","Data":"43573dd3ad95b159744c3552e2172f5cc9b9d1e790ae50bf02183e804ebbfb6f"} Jul 03 14:14:50.545518 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:50.545389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" event={"ID":"073d42fc-0af3-4b58-8006-2b4145773712","Type":"ContainerStarted","Data":"8459aef5bbb8e292b5d220b67e2c4422341cc9e627cfcf1f2c89d4e72f419fb9"} Jul 03 14:14:50.545518 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:50.545494 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:50.586988 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:50.586941 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" podStartSLOduration=1.006314291 podStartE2EDuration="5.586923406s" totalImagesPullingTime="4.580609115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:14:50.585756852 +0000 UTC m=+384.007521333" watchObservedRunningTime="2026-07-03 14:14:50.586923406 +0000 UTC m=+384.008687887" Jul 03 14:14:52.555119 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:52.555040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerStarted","Data":"5fe11e0bba235ca47a76a16c9baf63b7132adeb4194fc5ba922becba3741399b"} Jul 03 14:14:52.963212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:52.963183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:53.025692 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.025669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:53.128111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.128086 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:14:53.323341 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.323314 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b9n9z"] Jul 03 14:14:53.326277 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.326259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.328659 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.328644 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9dsgv\"" Jul 03 14:14:53.335401 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.335382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b9n9z"] Jul 03 14:14:53.417961 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.417923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-bound-sa-token\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.418143 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.417985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwdcw\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-kube-api-access-kwdcw\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.518357 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.518319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-bound-sa-token\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.518523 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.518385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwdcw\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-kube-api-access-kwdcw\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.528889 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.528863 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-bound-sa-token\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.529017 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.528998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwdcw\" (UniqueName: \"kubernetes.io/projected/a340dc84-6356-440e-aa95-c2ed7521cf98-kube-api-access-kwdcw\") pod \"cert-manager-55b4d954d5-b9n9z\" (UID: \"a340dc84-6356-440e-aa95-c2ed7521cf98\") " pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.559653 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.559622 2580 generic.go:356] "Generic (PLEG): container finished" podID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerID="5fe11e0bba235ca47a76a16c9baf63b7132adeb4194fc5ba922becba3741399b" exitCode=0 Jul 03 14:14:53.560048 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.559674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerDied","Data":"5fe11e0bba235ca47a76a16c9baf63b7132adeb4194fc5ba922becba3741399b"} Jul 03 14:14:53.635455 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.635382 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b9n9z" Jul 03 14:14:53.762266 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:53.762241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b9n9z"] Jul 03 14:14:53.764548 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:14:53.764517 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda340dc84_6356_440e_aa95_c2ed7521cf98.slice/crio-9580dcc852f226456404d33cbf0306aeb57b928aeaecf1a0f894d5765d6e1fa6 WatchSource:0}: Error finding container 9580dcc852f226456404d33cbf0306aeb57b928aeaecf1a0f894d5765d6e1fa6: Status 404 returned error can't find the container with id 9580dcc852f226456404d33cbf0306aeb57b928aeaecf1a0f894d5765d6e1fa6 Jul 03 14:14:54.564148 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:54.564109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b9n9z" event={"ID":"a340dc84-6356-440e-aa95-c2ed7521cf98","Type":"ContainerStarted","Data":"0fbc1faecd55846603be2621582dad71190f27302bbd4ac842e5b7644ccce4b6"} Jul 03 14:14:54.564148 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:54.564154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b9n9z" event={"ID":"a340dc84-6356-440e-aa95-c2ed7521cf98","Type":"ContainerStarted","Data":"9580dcc852f226456404d33cbf0306aeb57b928aeaecf1a0f894d5765d6e1fa6"} Jul 03 14:14:56.552671 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:56.552631 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-whbcm" Jul 03 14:14:56.574346 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:14:56.574290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-b9n9z" podStartSLOduration=3.574275079 podStartE2EDuration="3.574275079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:14:54.582570119 +0000 UTC m=+388.004334599" watchObservedRunningTime="2026-07-03 14:14:56.574275079 +0000 UTC m=+389.996039559" Jul 03 14:15:00.592930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:00.592901 2580 generic.go:356] "Generic (PLEG): container finished" podID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerID="673e23c7f803faa3806bb8443a813f22b947a5d65f5946e8bfe1ad53aec87839" exitCode=0 Jul 03 14:15:00.593283 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:00.592974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerDied","Data":"673e23c7f803faa3806bb8443a813f22b947a5d65f5946e8bfe1ad53aec87839"} Jul 03 14:15:01.724618 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.724593 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:15:01.783409 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.783376 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util\") pod \"9aeb6997-dad2-481c-8f8f-000359703cfe\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " Jul 03 14:15:01.783587 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.783420 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle\") pod \"9aeb6997-dad2-481c-8f8f-000359703cfe\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " Jul 03 14:15:01.783587 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.783470 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ccpp\" (UniqueName: \"kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp\") pod \"9aeb6997-dad2-481c-8f8f-000359703cfe\" (UID: \"9aeb6997-dad2-481c-8f8f-000359703cfe\") " Jul 03 14:15:01.783864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.783839 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle" (OuterVolumeSpecName: "bundle") pod "9aeb6997-dad2-481c-8f8f-000359703cfe" (UID: "9aeb6997-dad2-481c-8f8f-000359703cfe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:15:01.785671 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.785647 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp" (OuterVolumeSpecName: "kube-api-access-7ccpp") pod "9aeb6997-dad2-481c-8f8f-000359703cfe" (UID: "9aeb6997-dad2-481c-8f8f-000359703cfe"). InnerVolumeSpecName "kube-api-access-7ccpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:15:01.787713 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.787689 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util" (OuterVolumeSpecName: "util") pod "9aeb6997-dad2-481c-8f8f-000359703cfe" (UID: "9aeb6997-dad2-481c-8f8f-000359703cfe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:15:01.884701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.884612 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-util\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:15:01.884701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.884643 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9aeb6997-dad2-481c-8f8f-000359703cfe-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:15:01.884701 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:01.884653 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ccpp\" (UniqueName: \"kubernetes.io/projected/9aeb6997-dad2-481c-8f8f-000359703cfe-kube-api-access-7ccpp\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:15:02.601841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:02.601805 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" event={"ID":"9aeb6997-dad2-481c-8f8f-000359703cfe","Type":"ContainerDied","Data":"f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449"} Jul 03 14:15:02.601841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:02.601830 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fnkf9m" Jul 03 14:15:02.602053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:02.601876 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8b4dbea96715c7c67370bf02d1ce58848736182059a745d47e97383a3739449" Jul 03 14:15:04.953930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:04.953896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:05.054323 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:05.054288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:07.005796 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.005763 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9"] Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006028 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="extract" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006040 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="extract" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006053 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="util" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006060 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="util" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006073 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="pull" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006078 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="pull" Jul 03 14:15:07.006217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.006122 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aeb6997-dad2-481c-8f8f-000359703cfe" containerName="extract" Jul 03 14:15:07.074561 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.074523 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9"] Jul 03 14:15:07.074752 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.074616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.077381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.077355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 03 14:15:07.078115 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.078099 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 03 14:15:07.078115 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.078108 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-l7q7q\"" Jul 03 14:15:07.222269 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.222236 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.222269 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.222273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l65xn\" (UniqueName: \"kubernetes.io/projected/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-kube-api-access-l65xn\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.323377 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.323342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.323547 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.323396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l65xn\" (UniqueName: \"kubernetes.io/projected/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-kube-api-access-l65xn\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.323721 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.323697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.333427 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.333407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l65xn\" (UniqueName: \"kubernetes.io/projected/dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548-kube-api-access-l65xn\") pod \"openshift-lws-operator-fd8ccff4c-xc7d9\" (UID: \"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.383902 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.383875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" Jul 03 14:15:07.524833 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.524803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9"] Jul 03 14:15:07.528416 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:07.528389 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbd1d639_fc4a_4bbd_a2a8_8ec4bc2d7548.slice/crio-d271088cc8373991db553169175808243f569f59023621c6c72e0bf5fbb9735d WatchSource:0}: Error finding container d271088cc8373991db553169175808243f569f59023621c6c72e0bf5fbb9735d: Status 404 returned error can't find the container with id d271088cc8373991db553169175808243f569f59023621c6c72e0bf5fbb9735d Jul 03 14:15:07.620529 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:07.620448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" event={"ID":"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548","Type":"ContainerStarted","Data":"d271088cc8373991db553169175808243f569f59023621c6c72e0bf5fbb9735d"} Jul 03 14:15:11.637145 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:11.637114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" event={"ID":"dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548","Type":"ContainerStarted","Data":"ee07775f381fc17501be49e426dd5ffb959aa399d48b2f691224d2f468fe05b6"} Jul 03 14:15:11.662747 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:11.662673 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xc7d9" podStartSLOduration=2.5015776499999998 podStartE2EDuration="5.662657996s" totalImagesPullingTime="3.161080346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:11.662000319 +0000 UTC m=+405.083764801" watchObservedRunningTime="2026-07-03 14:15:11.662657996 +0000 UTC m=+405.084422504" Jul 03 14:15:12.399526 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:12.399492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:12.517376 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:12.517335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:14.334927 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:14.334883 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:18.581876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:18.581843 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:18.684983 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:18.684948 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:27.262608 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:27.262577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:27.362624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:27.362593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:27.866767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:27.866715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.154178 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.154104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.263339 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.263309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.293032 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.293000 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4"] Jul 03 14:15:28.295096 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.295078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.297907 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.297889 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 14:15:28.298056 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.298040 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 14:15:28.298328 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.298311 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 14:15:28.298563 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.298547 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 14:15:28.298637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.298608 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xwgf7\"" Jul 03 14:15:28.313939 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.313914 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4"] Jul 03 14:15:28.381800 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.381776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.381944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.381807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.381944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.381827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdv2\" (UniqueName: \"kubernetes.io/projected/20667d3d-6d30-4112-b1f8-f01fec9540b0-kube-api-access-2xdv2\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.381944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.381901 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.381944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.381921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483553 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483553 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483553 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdv2\" (UniqueName: \"kubernetes.io/projected/20667d3d-6d30-4112-b1f8-f01fec9540b0-kube-api-access-2xdv2\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.483853 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.483825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.485680 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.485651 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20667d3d-6d30-4112-b1f8-f01fec9540b0-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.485822 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.485800 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.485941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.485924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20667d3d-6d30-4112-b1f8-f01fec9540b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.516992 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.516962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdv2\" (UniqueName: \"kubernetes.io/projected/20667d3d-6d30-4112-b1f8-f01fec9540b0-kube-api-access-2xdv2\") pod \"opendatahub-operator-controller-manager-977f7bbf6-84bm4\" (UID: \"20667d3d-6d30-4112-b1f8-f01fec9540b0\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.604643 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.604605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:28.638836 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.638811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.739913 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.739816 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.751458 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.751436 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4"] Jul 03 14:15:28.753595 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:28.753571 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20667d3d_6d30_4112_b1f8_f01fec9540b0.slice/crio-5ba41b25467b50db2e90996347362c77b35f801a2dc527b38d52969704455eb7 WatchSource:0}: Error finding container 5ba41b25467b50db2e90996347362c77b35f801a2dc527b38d52969704455eb7: Status 404 returned error can't find the container with id 5ba41b25467b50db2e90996347362c77b35f801a2dc527b38d52969704455eb7 Jul 03 14:15:28.843848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.843816 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:28.939356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:28.939326 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:29.039572 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:29.039486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:29.705052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:29.705010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" event={"ID":"20667d3d-6d30-4112-b1f8-f01fec9540b0","Type":"ContainerStarted","Data":"5ba41b25467b50db2e90996347362c77b35f801a2dc527b38d52969704455eb7"} Jul 03 14:15:32.719150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:32.719115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" event={"ID":"20667d3d-6d30-4112-b1f8-f01fec9540b0","Type":"ContainerStarted","Data":"d4a9c7d0c4a70a1641933b605a81a1b2d95f5115bd65f2e43aa8cb0fc3ba22c6"} Jul 03 14:15:33.724162 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:33.724119 2580 generic.go:356] "Generic (PLEG): container finished" podID="20667d3d-6d30-4112-b1f8-f01fec9540b0" containerID="d4a9c7d0c4a70a1641933b605a81a1b2d95f5115bd65f2e43aa8cb0fc3ba22c6" exitCode=0 Jul 03 14:15:33.724532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:33.724167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" event={"ID":"20667d3d-6d30-4112-b1f8-f01fec9540b0","Type":"ContainerDied","Data":"d4a9c7d0c4a70a1641933b605a81a1b2d95f5115bd65f2e43aa8cb0fc3ba22c6"} Jul 03 14:15:34.729953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:34.729917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" event={"ID":"20667d3d-6d30-4112-b1f8-f01fec9540b0","Type":"ContainerStarted","Data":"975477d617cf73df01338c65eb9cbdba8fba65b7e83bebdd07c965b7bbc9cb6d"} Jul 03 14:15:34.730437 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:34.730057 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:34.758641 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:34.758588 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" podStartSLOduration=2.432915499 podStartE2EDuration="6.758576606s" totalImagesPullingTime="3.29977671s" totalInitContainerRuntime="1.025884397s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:34.757247206 +0000 UTC m=+428.179011699" watchObservedRunningTime="2026-07-03 14:15:34.758576606 +0000 UTC m=+428.180341086" Jul 03 14:15:45.736115 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:45.736082 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-84bm4" Jul 03 14:15:47.624071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:47.624041 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:47.742644 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:47.742611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:48.389092 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.389058 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:48.426337 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.426306 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-7zztf"] Jul 03 14:15:48.431509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.431491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.434064 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.434039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.434158 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.434104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tjnl\" (UniqueName: \"kubernetes.io/projected/c712a588-b3cb-47a2-aa9d-7b5432602677-kube-api-access-5tjnl\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.434158 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.434134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c712a588-b3cb-47a2-aa9d-7b5432602677-tmp\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.434503 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.434490 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 03 14:15:48.435257 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.435155 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 14:15:48.435257 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.435215 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 14:15:48.435481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.435463 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 03 14:15:48.449679 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.449657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-7zztf"] Jul 03 14:15:48.534865 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.534830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tjnl\" (UniqueName: \"kubernetes.io/projected/c712a588-b3cb-47a2-aa9d-7b5432602677-kube-api-access-5tjnl\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.535054 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.534876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c712a588-b3cb-47a2-aa9d-7b5432602677-tmp\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.535054 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.534926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.535054 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:48.535036 2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 03 14:15:48.535216 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:48.535113 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs podName:c712a588-b3cb-47a2-aa9d-7b5432602677 nodeName:}" failed. No retries permitted until 2026-07-03 14:15:49.035091725 +0000 UTC m=+442.456856184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs") pod "kube-auth-proxy-557657c976-7zztf" (UID: "c712a588-b3cb-47a2-aa9d-7b5432602677") : secret "kube-auth-proxy-tls" not found Jul 03 14:15:48.537146 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.537119 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c712a588-b3cb-47a2-aa9d-7b5432602677-tmp\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.545044 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.545021 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pk4xg"] Jul 03 14:15:48.546986 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.546972 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.553246 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.553227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 03 14:15:48.553424 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.553395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4psp6\"" Jul 03 14:15:48.557134 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.557114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tjnl\" (UniqueName: \"kubernetes.io/projected/c712a588-b3cb-47a2-aa9d-7b5432602677-kube-api-access-5tjnl\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:48.572481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.572461 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pk4xg"] Jul 03 14:15:48.628825 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.628794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:48.635914 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.635897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.635996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.635923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zntvp\" (UniqueName: \"kubernetes.io/projected/80c17a24-6f18-4c53-b19d-6ccddc379795-kube-api-access-zntvp\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.736480 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.736406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.736480 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.736436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zntvp\" (UniqueName: \"kubernetes.io/projected/80c17a24-6f18-4c53-b19d-6ccddc379795-kube-api-access-zntvp\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.736654 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:48.736547 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 03 14:15:48.736654 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:48.736609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert podName:80c17a24-6f18-4c53-b19d-6ccddc379795 nodeName:}" failed. No retries permitted until 2026-07-03 14:15:49.236592143 +0000 UTC m=+442.658356619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert") pod "odh-model-controller-7998c67dc4-pk4xg" (UID: "80c17a24-6f18-4c53-b19d-6ccddc379795") : secret "odh-model-controller-webhook-cert" not found Jul 03 14:15:48.752248 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.752224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zntvp\" (UniqueName: \"kubernetes.io/projected/80c17a24-6f18-4c53-b19d-6ccddc379795-kube-api-access-zntvp\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:48.780348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.780318 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:48.878915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:48.878847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:49.038462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.038376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:49.040766 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.040741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c712a588-b3cb-47a2-aa9d-7b5432602677-tls-certs\") pod \"kube-auth-proxy-557657c976-7zztf\" (UID: \"c712a588-b3cb-47a2-aa9d-7b5432602677\") " pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:49.240264 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.240225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:49.242625 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.242600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80c17a24-6f18-4c53-b19d-6ccddc379795-cert\") pod \"odh-model-controller-7998c67dc4-pk4xg\" (UID: \"80c17a24-6f18-4c53-b19d-6ccddc379795\") " pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:49.297784 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.297707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:49.340220 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.340192 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" Jul 03 14:15:49.456340 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.456305 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:49.495528 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.495492 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-7zztf"] Jul 03 14:15:49.498248 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:49.498224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc712a588_b3cb_47a2_aa9d_7b5432602677.slice/crio-4ab555fb6bfe9cdda07662e50249cac5162330adfcaa51c1cb0bd074db9de0e6 WatchSource:0}: Error finding container 4ab555fb6bfe9cdda07662e50249cac5162330adfcaa51c1cb0bd074db9de0e6: Status 404 returned error can't find the container with id 4ab555fb6bfe9cdda07662e50249cac5162330adfcaa51c1cb0bd074db9de0e6 Jul 03 14:15:49.593642 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.593531 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pk4xg"] Jul 03 14:15:49.596151 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:49.596117 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c17a24_6f18_4c53_b19d_6ccddc379795.slice/crio-436229d5ff0d9888107d83b7d66e32bc726b0863d71fc8bbec3873470eef4c9b WatchSource:0}: Error finding container 436229d5ff0d9888107d83b7d66e32bc726b0863d71fc8bbec3873470eef4c9b: Status 404 returned error can't find the container with id 436229d5ff0d9888107d83b7d66e32bc726b0863d71fc8bbec3873470eef4c9b Jul 03 14:15:49.719179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.719143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:49.788537 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.788503 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" event={"ID":"80c17a24-6f18-4c53-b19d-6ccddc379795","Type":"ContainerStarted","Data":"436229d5ff0d9888107d83b7d66e32bc726b0863d71fc8bbec3873470eef4c9b"} Jul 03 14:15:49.789458 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.789438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" event={"ID":"c712a588-b3cb-47a2-aa9d-7b5432602677","Type":"ContainerStarted","Data":"4ab555fb6bfe9cdda07662e50249cac5162330adfcaa51c1cb0bd074db9de0e6"} Jul 03 14:15:49.813610 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.813580 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnfqt"] Jul 03 14:15:49.816360 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.816344 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:49.819715 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.819697 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2rmsc\"" Jul 03 14:15:49.819857 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.819841 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 03 14:15:49.836624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.836605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnfqt"] Jul 03 14:15:49.845531 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.845486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:49.845531 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.845516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlxsv\" (UniqueName: \"kubernetes.io/projected/04285a29-1f90-4799-af8f-1b700f09ccf2-kube-api-access-vlxsv\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:49.946447 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.946411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:49.946630 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.946465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxsv\" (UniqueName: \"kubernetes.io/projected/04285a29-1f90-4799-af8f-1b700f09ccf2-kube-api-access-vlxsv\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:49.946630 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:49.946562 2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 03 14:15:49.946767 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:15:49.946637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs podName:04285a29-1f90-4799-af8f-1b700f09ccf2 nodeName:}" failed. No retries permitted until 2026-07-03 14:15:50.446616795 +0000 UTC m=+443.868381253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs") pod "model-serving-api-54d89bd5fc-gnfqt" (UID: "04285a29-1f90-4799-af8f-1b700f09ccf2") : secret "model-serving-api-tls" not found Jul 03 14:15:49.958795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:49.958768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxsv\" (UniqueName: \"kubernetes.io/projected/04285a29-1f90-4799-af8f-1b700f09ccf2-kube-api-access-vlxsv\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:50.098655 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.098570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:50.107278 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.107246 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:15:50.110685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.110662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.113275 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.113249 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 03 14:15:50.113275 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.113264 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tphwm\"" Jul 03 14:15:50.113275 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.113271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 03 14:15:50.113708 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.113671 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 14:15:50.113825 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.113717 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 03 14:15:50.130023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.129992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:15:50.148445 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148577 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148577 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4m42\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.148839 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.148717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.195262 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.195231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:50.250029 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.249987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4m42\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.250892 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.250866 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.253345 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.253310 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.253445 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.253377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.253445 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.253377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.253566 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.253512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.259793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.259750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.259987 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.259969 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4m42\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42\") pod \"istiod-openshift-gateway-7574459656-sdflv\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.423741 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.423640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:50.453589 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.453556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:50.457003 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.456975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04285a29-1f90-4799-af8f-1b700f09ccf2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gnfqt\" (UID: \"04285a29-1f90-4799-af8f-1b700f09ccf2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:50.470013 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.469941 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:50.613836 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.613800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:15:50.619684 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:50.619358 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1601c5a6_0bdf_48d9_87af_813acc0710b0.slice/crio-5dfc1d0d087fb986f088ddb600a955b3f8c0be8b2c35180704c515d776b41de0 WatchSource:0}: Error finding container 5dfc1d0d087fb986f088ddb600a955b3f8c0be8b2c35180704c515d776b41de0: Status 404 returned error can't find the container with id 5dfc1d0d087fb986f088ddb600a955b3f8c0be8b2c35180704c515d776b41de0 Jul 03 14:15:50.726314 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.726230 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:50.800308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.800196 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" event={"ID":"1601c5a6-0bdf-48d9-87af-813acc0710b0","Type":"ContainerStarted","Data":"5dfc1d0d087fb986f088ddb600a955b3f8c0be8b2c35180704c515d776b41de0"} Jul 03 14:15:50.895312 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:50.895285 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gnfqt"] Jul 03 14:15:50.898674 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:15:50.898643 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04285a29_1f90_4799_af8f_1b700f09ccf2.slice/crio-ba858cf78429b1fc133b94e4aafc8688b16cef2a93a6ac4b3d64325d8e02dd1b WatchSource:0}: Error finding container ba858cf78429b1fc133b94e4aafc8688b16cef2a93a6ac4b3d64325d8e02dd1b: Status 404 returned error can't find the container with id ba858cf78429b1fc133b94e4aafc8688b16cef2a93a6ac4b3d64325d8e02dd1b Jul 03 14:15:51.420132 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:51.420096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:51.807507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:51.807427 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" event={"ID":"04285a29-1f90-4799-af8f-1b700f09ccf2","Type":"ContainerStarted","Data":"ba858cf78429b1fc133b94e4aafc8688b16cef2a93a6ac4b3d64325d8e02dd1b"} Jul 03 14:15:54.753190 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:54.753136 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:15:54.753447 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:54.753237 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:15:55.314031 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.313997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:55.422183 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.422153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:55.671793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.671763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:55.786417 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.786382 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:55.826623 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.826593 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" event={"ID":"80c17a24-6f18-4c53-b19d-6ccddc379795","Type":"ContainerStarted","Data":"8dbbe9bf8e2a493a20f7c78f104d5bbaf3bb4c1b2163e08525f5043d73de719c"} Jul 03 14:15:55.826824 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.826673 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:15:55.828278 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.828254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" event={"ID":"04285a29-1f90-4799-af8f-1b700f09ccf2","Type":"ContainerStarted","Data":"0513ad4996d7a661975f7948bde9f85836d6aadfb5999eef5ac19a783ec2766b"} Jul 03 14:15:55.828388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.828328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:15:55.829708 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.829684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" event={"ID":"1601c5a6-0bdf-48d9-87af-813acc0710b0","Type":"ContainerStarted","Data":"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67"} Jul 03 14:15:55.829835 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.829742 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:15:55.831196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.831172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" event={"ID":"c712a588-b3cb-47a2-aa9d-7b5432602677","Type":"ContainerStarted","Data":"8d7704a3050e50db6ba27602f4dc53845363d321b3d55667387d369b7014b8e1"} Jul 03 14:15:55.831783 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.831758 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-sdflv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 14:15:55.831885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.831809 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 14:15:55.864172 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.864124 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" podStartSLOduration=2.712006647 podStartE2EDuration="7.864110905s" totalImagesPullingTime="5.152104258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:55.863201811 +0000 UTC m=+449.284966293" watchObservedRunningTime="2026-07-03 14:15:55.864110905 +0000 UTC m=+449.285875364" Jul 03 14:15:55.912073 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.912023 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" podStartSLOduration=1.7812371310000001 podStartE2EDuration="5.912007224s" totalImagesPullingTime="4.130770093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:55.910672007 +0000 UTC m=+449.332436488" watchObservedRunningTime="2026-07-03 14:15:55.912007224 +0000 UTC m=+449.333771683" Jul 03 14:15:55.971235 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:55.971125 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-557657c976-7zztf" podStartSLOduration=2.7284501839999997 podStartE2EDuration="7.971094609s" totalImagesPullingTime="5.242644425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:55.971029635 +0000 UTC m=+449.392794118" watchObservedRunningTime="2026-07-03 14:15:55.971094609 +0000 UTC m=+449.392859092" Jul 03 14:15:56.011063 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:56.010999 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" podStartSLOduration=3.157955668 podStartE2EDuration="7.010974728s" totalImagesPullingTime="3.85301906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:56.006927634 +0000 UTC m=+449.428692116" watchObservedRunningTime="2026-07-03 14:15:56.010974728 +0000 UTC m=+449.432739215" Jul 03 14:15:56.291545 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:56.291437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:15:56.836066 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:15:56.836036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:16:00.113525 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.113487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:00.314762 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.314718 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh"] Jul 03 14:16:00.317562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.317546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.326122 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.326105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 03 14:16:00.326205 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.326147 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nlfkk\"" Jul 03 14:16:00.369129 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.369061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh"] Jul 03 14:16:00.371645 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.371616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:00.399207 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.399176 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm"] Jul 03 14:16:00.402265 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.402249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.406613 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.406592 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jkvh7\"" Jul 03 14:16:00.406712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.406665 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 03 14:16:00.439375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.439349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.439479 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.439387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29015242-3f99-4879-8658-7acf80cfa262-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.439479 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.439417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czd6t\" (UniqueName: \"kubernetes.io/projected/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-kube-api-access-czd6t\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.439479 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.439461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvq4q\" (UniqueName: \"kubernetes.io/projected/29015242-3f99-4879-8658-7acf80cfa262-kube-api-access-lvq4q\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.460184 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.460160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm"] Jul 03 14:16:00.540811 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.540776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.540963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.540824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29015242-3f99-4879-8658-7acf80cfa262-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.540963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.540857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czd6t\" (UniqueName: \"kubernetes.io/projected/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-kube-api-access-czd6t\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.540963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.540893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq4q\" (UniqueName: \"kubernetes.io/projected/29015242-3f99-4879-8658-7acf80cfa262-kube-api-access-lvq4q\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.540963 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:16:00.540945 2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 03 14:16:00.541208 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:16:00.541010 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert podName:b55c6133-29ef-43d6-b6b2-cc5a2e3038a1 nodeName:}" failed. No retries permitted until 2026-07-03 14:16:01.040989116 +0000 UTC m=+454.462753575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert") pod "kserve-controller-manager-d6f4b65b8-sv2nm" (UID: "b55c6133-29ef-43d6-b6b2-cc5a2e3038a1") : secret "kserve-webhook-server-cert" not found Jul 03 14:16:00.543229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.543207 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29015242-3f99-4879-8658-7acf80cfa262-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.603485 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.599476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czd6t\" (UniqueName: \"kubernetes.io/projected/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-kube-api-access-czd6t\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:00.603485 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.599673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvq4q\" (UniqueName: \"kubernetes.io/projected/29015242-3f99-4879-8658-7acf80cfa262-kube-api-access-lvq4q\") pod \"llmisvc-controller-manager-777c6bd7d5-6tpqh\" (UID: \"29015242-3f99-4879-8658-7acf80cfa262\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.623969 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.623873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:00.626833 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.626812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:00.837891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.837862 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh"] Jul 03 14:16:00.838632 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:16:00.838598 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod29015242_3f99_4879_8658_7acf80cfa262.slice/crio-cfbd8eae9a6502243cb0b475ac9a92a9ef08d125872f81c07c586830f208d634 WatchSource:0}: Error finding container cfbd8eae9a6502243cb0b475ac9a92a9ef08d125872f81c07c586830f208d634: Status 404 returned error can't find the container with id cfbd8eae9a6502243cb0b475ac9a92a9ef08d125872f81c07c586830f208d634 Jul 03 14:16:00.851858 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.851830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" event={"ID":"29015242-3f99-4879-8658-7acf80cfa262","Type":"ContainerStarted","Data":"cfbd8eae9a6502243cb0b475ac9a92a9ef08d125872f81c07c586830f208d634"} Jul 03 14:16:00.906990 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:00.906881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:01.046315 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.046269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:01.048670 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.048648 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b55c6133-29ef-43d6-b6b2-cc5a2e3038a1-cert\") pod \"kserve-controller-manager-d6f4b65b8-sv2nm\" (UID: \"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:01.311548 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.311471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:01.491337 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.491297 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm"] Jul 03 14:16:01.573636 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:16:01.573605 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55c6133_29ef_43d6_b6b2_cc5a2e3038a1.slice/crio-8f0d2d06582ada37f8bcf44d1142496ba0a1cf6aed69657cc7e18318c976abcf WatchSource:0}: Error finding container 8f0d2d06582ada37f8bcf44d1142496ba0a1cf6aed69657cc7e18318c976abcf: Status 404 returned error can't find the container with id 8f0d2d06582ada37f8bcf44d1142496ba0a1cf6aed69657cc7e18318c976abcf Jul 03 14:16:01.787571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.787541 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:01.858426 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.858353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" event={"ID":"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1","Type":"ContainerStarted","Data":"8f0d2d06582ada37f8bcf44d1142496ba0a1cf6aed69657cc7e18318c976abcf"} Jul 03 14:16:01.898015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:01.897984 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:02.191878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.191797 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:16:02.192055 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.192028 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerName="discovery" containerID="cri-o://77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67" gracePeriod=30 Jul 03 14:16:02.473449 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.473423 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:16:02.560391 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560356 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4m42\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560416 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560450 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560485 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560511 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560562 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.560873 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.560593 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts\") pod \"1601c5a6-0bdf-48d9-87af-813acc0710b0\" (UID: \"1601c5a6-0bdf-48d9-87af-813acc0710b0\") " Jul 03 14:16:02.561441 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.561403 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:16:02.564008 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.563975 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts" (OuterVolumeSpecName: "cacerts") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:16:02.564168 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.564029 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42" (OuterVolumeSpecName: "kube-api-access-d4m42") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "kube-api-access-d4m42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:16:02.565304 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.565276 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs" (OuterVolumeSpecName: "local-certs") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:16:02.565678 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.565641 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:16:02.566415 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.566379 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:16:02.566507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.566483 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1601c5a6-0bdf-48d9-87af-813acc0710b0" (UID: "1601c5a6-0bdf-48d9-87af-813acc0710b0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:16:02.662239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662201 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4m42\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-kube-api-access-d4m42\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662231 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-dns-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662245 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-csr-ca-configmap\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662261 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662275 2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1601c5a6-0bdf-48d9-87af-813acc0710b0-local-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662287 2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-istio-kubeconfig\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.662473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.662300 2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1601c5a6-0bdf-48d9-87af-813acc0710b0-cacerts\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:02.864671 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.864629 2580 generic.go:356] "Generic (PLEG): container finished" podID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerID="77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67" exitCode=0 Jul 03 14:16:02.864864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.864679 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" Jul 03 14:16:02.864864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.864695 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" event={"ID":"1601c5a6-0bdf-48d9-87af-813acc0710b0","Type":"ContainerDied","Data":"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67"} Jul 03 14:16:02.864864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.864762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-sdflv" event={"ID":"1601c5a6-0bdf-48d9-87af-813acc0710b0","Type":"ContainerDied","Data":"5dfc1d0d087fb986f088ddb600a955b3f8c0be8b2c35180704c515d776b41de0"} Jul 03 14:16:02.864864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.864786 2580 scope.go:117] "RemoveContainer" containerID="77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67" Jul 03 14:16:02.873484 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.873469 2580 scope.go:117] "RemoveContainer" containerID="77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67" Jul 03 14:16:02.873780 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:16:02.873760 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67\": container with ID starting with 77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67 not found: ID does not exist" containerID="77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67" Jul 03 14:16:02.873833 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.873789 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67"} err="failed to get container status \"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67\": rpc error: code = NotFound desc = could not find container \"77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67\": container with ID starting with 77dcc5b5b5c510af30124fff95bb6589fc67beabfbdecc3670789747eb781b67 not found: ID does not exist" Jul 03 14:16:02.914778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.914743 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:16:02.952428 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:02.952398 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-sdflv"] Jul 03 14:16:03.205436 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:03.205354 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" path="/var/lib/kubelet/pods/1601c5a6-0bdf-48d9-87af-813acc0710b0/volumes" Jul 03 14:16:04.875888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.875848 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" event={"ID":"29015242-3f99-4879-8658-7acf80cfa262","Type":"ContainerStarted","Data":"10cf481d14366599db41e176d614f21e9f085b3e4c7afcb601f21e44fdf56cfa"} Jul 03 14:16:04.876284 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.875960 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:04.877462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.877440 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" event={"ID":"b55c6133-29ef-43d6-b6b2-cc5a2e3038a1","Type":"ContainerStarted","Data":"241cef58eda2707a0e136b8402d69ec275a96715135967baf5b923c710752209"} Jul 03 14:16:04.877576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.877557 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:04.901792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.901744 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" podStartSLOduration=1.1878490290000001 podStartE2EDuration="4.901712853s" totalImagesPullingTime="3.713863824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:16:04.900068203 +0000 UTC m=+458.321832684" watchObservedRunningTime="2026-07-03 14:16:04.901712853 +0000 UTC m=+458.323477334" Jul 03 14:16:04.922885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:04.922846 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" podStartSLOduration=1.942734144 podStartE2EDuration="4.922834845s" totalImagesPullingTime="2.980100701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:16:04.921892478 +0000 UTC m=+458.343656970" watchObservedRunningTime="2026-07-03 14:16:04.922834845 +0000 UTC m=+458.344599326" Jul 03 14:16:06.837644 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:06.837614 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pk4xg" Jul 03 14:16:06.839652 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:06.839629 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gnfqt" Jul 03 14:16:26.435640 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:26.435603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:26.511557 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:26.511519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:27.225324 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:27.225283 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:28.377565 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:28.377534 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:28.477448 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:28.477418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:35.884890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:35.884858 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6tpqh" Jul 03 14:16:35.887904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:35.887882 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sv2nm" Jul 03 14:16:36.886593 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.886556 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d86bb67dd-27tqp"] Jul 03 14:16:36.886997 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.886851 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerName="discovery" Jul 03 14:16:36.886997 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.886862 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerName="discovery" Jul 03 14:16:36.886997 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.886923 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1601c5a6-0bdf-48d9-87af-813acc0710b0" containerName="discovery" Jul 03 14:16:36.951970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.951936 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d86bb67dd-27tqp"] Jul 03 14:16:36.952167 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.952050 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:36.978976 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:36.978949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:37.046984 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.046950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-service-ca\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.046984 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.046988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.047196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.047008 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.047196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.047047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-trusted-ca-bundle\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.047196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.047075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpvcx\" (UniqueName: \"kubernetes.io/projected/ebdaad94-090d-49b5-bae4-f9f354706dcd-kube-api-access-gpvcx\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.047196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.047098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-oauth-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.047196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.047135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-oauth-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.063091 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.063060 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:37.149234 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-service-ca\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.149465 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.149576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.149694 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-trusted-ca-bundle\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.149841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpvcx\" (UniqueName: \"kubernetes.io/projected/ebdaad94-090d-49b5-bae4-f9f354706dcd-kube-api-access-gpvcx\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.149950 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.149937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-oauth-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.150098 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.150083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-oauth-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.150971 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.150947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-oauth-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.151745 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.151707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-service-ca\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.154689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.154647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.155424 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.155397 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-serving-cert\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.155518 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.155400 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebdaad94-090d-49b5-bae4-f9f354706dcd-trusted-ca-bundle\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.156776 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.156752 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ebdaad94-090d-49b5-bae4-f9f354706dcd-console-oauth-config\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.165252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.165221 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:37.166156 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.166132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpvcx\" (UniqueName: \"kubernetes.io/projected/ebdaad94-090d-49b5-bae4-f9f354706dcd-kube-api-access-gpvcx\") pod \"console-7d86bb67dd-27tqp\" (UID: \"ebdaad94-090d-49b5-bae4-f9f354706dcd\") " pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.261097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.261065 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:37.420829 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.420805 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d86bb67dd-27tqp"] Jul 03 14:16:37.423184 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:16:37.423161 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebdaad94_090d_49b5_bae4_f9f354706dcd.slice/crio-1eecd8d117b17ce035964e81a82dc157554d22ab5cdad86f4fa65adf4becc720 WatchSource:0}: Error finding container 1eecd8d117b17ce035964e81a82dc157554d22ab5cdad86f4fa65adf4becc720: Status 404 returned error can't find the container with id 1eecd8d117b17ce035964e81a82dc157554d22ab5cdad86f4fa65adf4becc720 Jul 03 14:16:37.684500 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.684414 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:37.789003 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:37.788972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:38.017377 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:38.017286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d86bb67dd-27tqp" event={"ID":"ebdaad94-090d-49b5-bae4-f9f354706dcd","Type":"ContainerStarted","Data":"b1b12488441058f6a0ea777e55ef01d27c02a3b519307f40b5060118c91a1338"} Jul 03 14:16:38.017377 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:38.017341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d86bb67dd-27tqp" event={"ID":"ebdaad94-090d-49b5-bae4-f9f354706dcd","Type":"ContainerStarted","Data":"1eecd8d117b17ce035964e81a82dc157554d22ab5cdad86f4fa65adf4becc720"} Jul 03 14:16:38.043659 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:38.043606 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d86bb67dd-27tqp" podStartSLOduration=2.043591181 podStartE2EDuration="2.043591181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:16:38.041669772 +0000 UTC m=+491.463434256" watchObservedRunningTime="2026-07-03 14:16:38.043591181 +0000 UTC m=+491.465355662" Jul 03 14:16:47.262087 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:47.262051 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:47.262654 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:47.262115 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:47.266862 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:47.266840 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:48.063172 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:48.063144 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d86bb67dd-27tqp" Jul 03 14:16:48.117842 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:48.117806 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:16:48.244035 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:48.243996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:48.345256 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:48.345227 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:48.471508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:48.471478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:49.395429 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.395395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:49.534462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.534430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:49.535097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.535074 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:49.538495 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.538475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:49.543247 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.543228 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 14:16:49.543358 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.543258 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 14:16:49.543358 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.543321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8zr5n\"" Jul 03 14:16:49.554425 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.554401 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:49.632512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.632483 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:16:49.647705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.647645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp4mb\" (UniqueName: \"kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb\") pod \"kuadrant-operator-catalog-dm6kx\" (UID: \"5cf74afb-804e-4776-ac95-ee5482154662\") " pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:49.748843 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.748812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp4mb\" (UniqueName: \"kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb\") pod \"kuadrant-operator-catalog-dm6kx\" (UID: \"5cf74afb-804e-4776-ac95-ee5482154662\") " pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:49.757346 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.757322 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp4mb\" (UniqueName: \"kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb\") pod \"kuadrant-operator-catalog-dm6kx\" (UID: \"5cf74afb-804e-4776-ac95-ee5482154662\") " pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:49.848129 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.848093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:49.852757 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:49.852708 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:50.041406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.041374 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:50.042594 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:16:50.042568 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf74afb_804e_4776_ac95_ee5482154662.slice/crio-d722e56e92fd51ab21cc319f0b8038d5736acab45a4db63cb7f22211026dca0c WatchSource:0}: Error finding container d722e56e92fd51ab21cc319f0b8038d5736acab45a4db63cb7f22211026dca0c: Status 404 returned error can't find the container with id d722e56e92fd51ab21cc319f0b8038d5736acab45a4db63cb7f22211026dca0c Jul 03 14:16:50.069295 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.069265 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" event={"ID":"5cf74afb-804e-4776-ac95-ee5482154662","Type":"ContainerStarted","Data":"d722e56e92fd51ab21cc319f0b8038d5736acab45a4db63cb7f22211026dca0c"} Jul 03 14:16:50.088532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.088505 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c55gk"] Jul 03 14:16:50.092365 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.092349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:16:50.156196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.156165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdlsn\" (UniqueName: \"kubernetes.io/projected/f669de20-7738-4a34-8010-e824aa940d01-kube-api-access-sdlsn\") pod \"kuadrant-operator-catalog-c55gk\" (UID: \"f669de20-7738-4a34-8010-e824aa940d01\") " pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:16:50.173916 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.173846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c55gk"] Jul 03 14:16:50.257338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.257300 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdlsn\" (UniqueName: \"kubernetes.io/projected/f669de20-7738-4a34-8010-e824aa940d01-kube-api-access-sdlsn\") pod \"kuadrant-operator-catalog-c55gk\" (UID: \"f669de20-7738-4a34-8010-e824aa940d01\") " pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:16:50.274286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.274254 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdlsn\" (UniqueName: \"kubernetes.io/projected/f669de20-7738-4a34-8010-e824aa940d01-kube-api-access-sdlsn\") pod \"kuadrant-operator-catalog-c55gk\" (UID: \"f669de20-7738-4a34-8010-e824aa940d01\") " pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:16:50.401970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.401930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:16:50.538297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:50.538263 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c55gk"] Jul 03 14:16:50.542125 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:16:50.542099 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf669de20_7738_4a34_8010_e824aa940d01.slice/crio-6a9d606bd5ff96ffe8d716ae517bfe4d9c6060fd58345bd5ed7b4d42dbefdb69 WatchSource:0}: Error finding container 6a9d606bd5ff96ffe8d716ae517bfe4d9c6060fd58345bd5ed7b4d42dbefdb69: Status 404 returned error can't find the container with id 6a9d606bd5ff96ffe8d716ae517bfe4d9c6060fd58345bd5ed7b4d42dbefdb69 Jul 03 14:16:51.075428 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:51.075399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" event={"ID":"f669de20-7738-4a34-8010-e824aa940d01","Type":"ContainerStarted","Data":"6a9d606bd5ff96ffe8d716ae517bfe4d9c6060fd58345bd5ed7b4d42dbefdb69"} Jul 03 14:16:53.086473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.086431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" event={"ID":"f669de20-7738-4a34-8010-e824aa940d01","Type":"ContainerStarted","Data":"0cf734c605e035ee983d876ce841848f5ba84f93743b6d618d84ac4fb9a33e97"} Jul 03 14:16:53.087878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.087853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" event={"ID":"5cf74afb-804e-4776-ac95-ee5482154662","Type":"ContainerStarted","Data":"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e"} Jul 03 14:16:53.088005 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.087903 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" podUID="5cf74afb-804e-4776-ac95-ee5482154662" containerName="registry-server" containerID="cri-o://72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e" gracePeriod=2 Jul 03 14:16:53.148808 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.148751 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" podStartSLOduration=1.764465438 podStartE2EDuration="4.148709615s" totalImagesPullingTime="2.384244177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:16:53.147281327 +0000 UTC m=+506.569045808" watchObservedRunningTime="2026-07-03 14:16:53.148709615 +0000 UTC m=+506.570474137" Jul 03 14:16:53.150187 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.150152 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" podStartSLOduration=1.262271902 podStartE2EDuration="3.150142846s" totalImagesPullingTime="1.887870944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:16:53.119171089 +0000 UTC m=+506.540935582" watchObservedRunningTime="2026-07-03 14:16:53.150142846 +0000 UTC m=+506.571907326" Jul 03 14:16:53.326044 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.326018 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:53.384547 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.384462 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp4mb\" (UniqueName: \"kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb\") pod \"5cf74afb-804e-4776-ac95-ee5482154662\" (UID: \"5cf74afb-804e-4776-ac95-ee5482154662\") " Jul 03 14:16:53.386571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.386541 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb" (OuterVolumeSpecName: "kube-api-access-gp4mb") pod "5cf74afb-804e-4776-ac95-ee5482154662" (UID: "5cf74afb-804e-4776-ac95-ee5482154662"). InnerVolumeSpecName "kube-api-access-gp4mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:16:53.485501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:53.485472 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp4mb\" (UniqueName: \"kubernetes.io/projected/5cf74afb-804e-4776-ac95-ee5482154662-kube-api-access-gp4mb\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:16:54.093231 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.093194 2580 generic.go:356] "Generic (PLEG): container finished" podID="5cf74afb-804e-4776-ac95-ee5482154662" containerID="72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e" exitCode=0 Jul 03 14:16:54.093705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.093239 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" Jul 03 14:16:54.093705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.093289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" event={"ID":"5cf74afb-804e-4776-ac95-ee5482154662","Type":"ContainerDied","Data":"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e"} Jul 03 14:16:54.093705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.093334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dm6kx" event={"ID":"5cf74afb-804e-4776-ac95-ee5482154662","Type":"ContainerDied","Data":"d722e56e92fd51ab21cc319f0b8038d5736acab45a4db63cb7f22211026dca0c"} Jul 03 14:16:54.093705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.093355 2580 scope.go:117] "RemoveContainer" containerID="72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e" Jul 03 14:16:54.102012 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.101995 2580 scope.go:117] "RemoveContainer" containerID="72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e" Jul 03 14:16:54.102301 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:16:54.102279 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e\": container with ID starting with 72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e not found: ID does not exist" containerID="72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e" Jul 03 14:16:54.102386 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.102309 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e"} err="failed to get container status \"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e\": rpc error: code = NotFound desc = could not find container \"72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e\": container with ID starting with 72bdc278b4285fd7b90968d8d13cab3c4130d8bca7d3e78d0d4eaef47aa2497e not found: ID does not exist" Jul 03 14:16:54.121871 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.121826 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:54.125131 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:54.125110 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dm6kx"] Jul 03 14:16:55.204789 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:16:55.204758 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cf74afb-804e-4776-ac95-ee5482154662" path="/var/lib/kubelet/pods/5cf74afb-804e-4776-ac95-ee5482154662/volumes" Jul 03 14:17:00.402354 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:00.402321 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:17:00.402891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:00.402376 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:17:00.424640 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:00.424616 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:17:01.146692 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:01.146667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-c55gk" Jul 03 14:17:05.008211 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.008174 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4"] Jul 03 14:17:05.008579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.008476 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cf74afb-804e-4776-ac95-ee5482154662" containerName="registry-server" Jul 03 14:17:05.008579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.008486 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cf74afb-804e-4776-ac95-ee5482154662" containerName="registry-server" Jul 03 14:17:05.008579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.008546 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cf74afb-804e-4776-ac95-ee5482154662" containerName="registry-server" Jul 03 14:17:05.013053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.013036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.019645 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.019622 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4p5v9\"" Jul 03 14:17:05.027126 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.027105 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4"] Jul 03 14:17:05.079233 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.079202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.079390 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.079246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68l7t\" (UniqueName: \"kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.079390 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.079304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.180365 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.180331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.180512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.180378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68l7t\" (UniqueName: \"kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.180512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.180422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.180702 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.180682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.180780 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.180761 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.197741 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.197707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68l7t\" (UniqueName: \"kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.281804 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.281705 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:05.322064 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.322034 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:05.465847 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.465815 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4"] Jul 03 14:17:05.469249 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:05.469224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409a5eb6_527d_4915_b6a2_b0a976726af0.slice/crio-7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9 WatchSource:0}: Error finding container 7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9: Status 404 returned error can't find the container with id 7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9 Jul 03 14:17:05.629092 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.626196 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7"] Jul 03 14:17:05.631160 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.631130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.650774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.650746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7"] Jul 03 14:17:05.684957 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.684934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.685099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.684984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf9mn\" (UniqueName: \"kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.685099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.685025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.785568 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.785535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf9mn\" (UniqueName: \"kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.785794 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.785590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.785794 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.785641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.786018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.785994 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.786084 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.786034 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.799442 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.799414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf9mn\" (UniqueName: \"kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:05.939408 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:05.939376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:06.021668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.021636 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8"] Jul 03 14:17:06.027862 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.027839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.038788 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.038702 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8"] Jul 03 14:17:06.077138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.076960 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7"] Jul 03 14:17:06.079610 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:06.079590 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcf8e70_fa87_4f7e_bd6a_35b4d84072cc.slice/crio-06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092 WatchSource:0}: Error finding container 06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092: Status 404 returned error can't find the container with id 06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092 Jul 03 14:17:06.088580 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.088550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zktqs\" (UniqueName: \"kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.088674 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.088601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.088674 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.088652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.148121 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.148086 2580 generic.go:356] "Generic (PLEG): container finished" podID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerID="a60e6930c0799dc17f718413c1fb91aa490a079df6b6af39cd5b3adf1124cfbe" exitCode=0 Jul 03 14:17:06.148229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.148164 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerDied","Data":"a60e6930c0799dc17f718413c1fb91aa490a079df6b6af39cd5b3adf1124cfbe"} Jul 03 14:17:06.148229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.148211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerStarted","Data":"7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9"} Jul 03 14:17:06.149886 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.149860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerStarted","Data":"57155419dadfb0f6810d928a2a2ff59758295b1683ed12e5c59794a1e7eaa961"} Jul 03 14:17:06.150021 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.149904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerStarted","Data":"06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092"} Jul 03 14:17:06.189472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.189417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zktqs\" (UniqueName: \"kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.189472 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.189464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.189664 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.189644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.189875 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.189859 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.189925 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.189904 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.210532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.210506 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zktqs\" (UniqueName: \"kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.340221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.340191 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:06.480474 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.480399 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b"] Jul 03 14:17:06.484710 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.484693 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.521040 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.521011 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b"] Jul 03 14:17:06.525238 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.525213 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8"] Jul 03 14:17:06.527883 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:06.527855 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc27fa0_cb5b_4f48_abfd_3784460fbc5c.slice/crio-d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7 WatchSource:0}: Error finding container d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7: Status 404 returned error can't find the container with id d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7 Jul 03 14:17:06.594363 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.594331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xqtq\" (UniqueName: \"kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.594475 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.594441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.594475 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.594461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.694868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.694834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.694868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.694871 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.695085 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.694910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xqtq\" (UniqueName: \"kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.695200 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.695184 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.695237 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.695207 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.722188 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.722155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xqtq\" (UniqueName: \"kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.793576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.793513 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:06.934289 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:06.934257 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b"] Jul 03 14:17:06.937640 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:06.937612 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56eabe35_4d42_43f3_9597_3688a32b8f62.slice/crio-6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c WatchSource:0}: Error finding container 6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c: Status 404 returned error can't find the container with id 6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c Jul 03 14:17:07.156629 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.156602 2580 generic.go:356] "Generic (PLEG): container finished" podID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerID="55cf8d11e4b73dd08e4f3143db4999cb4ae2936385126f5d41b364cc86de370b" exitCode=0 Jul 03 14:17:07.157020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.156678 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" event={"ID":"56eabe35-4d42-43f3-9597-3688a32b8f62","Type":"ContainerDied","Data":"55cf8d11e4b73dd08e4f3143db4999cb4ae2936385126f5d41b364cc86de370b"} Jul 03 14:17:07.157020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.156772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" event={"ID":"56eabe35-4d42-43f3-9597-3688a32b8f62","Type":"ContainerStarted","Data":"6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c"} Jul 03 14:17:07.158887 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.158863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerStarted","Data":"d501daeda36dabb6470da8d716b6b0918227eb92a06002284bb23cea2e63c471"} Jul 03 14:17:07.160392 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.160371 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerID="aafc172df0ed546dafbea670e1527c61d48fe2515f6d61add5ee3750019c2b6c" exitCode=0 Jul 03 14:17:07.160487 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.160447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" event={"ID":"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c","Type":"ContainerDied","Data":"aafc172df0ed546dafbea670e1527c61d48fe2515f6d61add5ee3750019c2b6c"} Jul 03 14:17:07.160533 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.160486 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" event={"ID":"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c","Type":"ContainerStarted","Data":"d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7"} Jul 03 14:17:07.162062 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.162034 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerID="57155419dadfb0f6810d928a2a2ff59758295b1683ed12e5c59794a1e7eaa961" exitCode=0 Jul 03 14:17:07.162142 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:07.162064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerDied","Data":"57155419dadfb0f6810d928a2a2ff59758295b1683ed12e5c59794a1e7eaa961"} Jul 03 14:17:08.168945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.168918 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerID="9d508ddc845391664944664ca65bc3c973739c2c74ea6e3be54afcbc55c8c321" exitCode=0 Jul 03 14:17:08.169306 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.169003 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" event={"ID":"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c","Type":"ContainerDied","Data":"9d508ddc845391664944664ca65bc3c973739c2c74ea6e3be54afcbc55c8c321"} Jul 03 14:17:08.170840 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.170816 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerID="0aa2c4d46843e85a392ec296ca5e810fad631f96a67a27c72b9e3e110026abed" exitCode=0 Jul 03 14:17:08.170937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.170901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerDied","Data":"0aa2c4d46843e85a392ec296ca5e810fad631f96a67a27c72b9e3e110026abed"} Jul 03 14:17:08.172870 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.172851 2580 generic.go:356] "Generic (PLEG): container finished" podID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerID="277f50422b7b20170a84b9b1834f0c112263268ffd11e1220590e2458e984013" exitCode=0 Jul 03 14:17:08.172964 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.172915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" event={"ID":"56eabe35-4d42-43f3-9597-3688a32b8f62","Type":"ContainerDied","Data":"277f50422b7b20170a84b9b1834f0c112263268ffd11e1220590e2458e984013"} Jul 03 14:17:08.175007 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.174990 2580 generic.go:356] "Generic (PLEG): container finished" podID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerID="d501daeda36dabb6470da8d716b6b0918227eb92a06002284bb23cea2e63c471" exitCode=0 Jul 03 14:17:08.175071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:08.175060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerDied","Data":"d501daeda36dabb6470da8d716b6b0918227eb92a06002284bb23cea2e63c471"} Jul 03 14:17:09.181235 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.181202 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerID="c988bcb03bd69b3c67b21ba2a40e4a78f3f2d3e1d53e3c23489c88f0e92f9b67" exitCode=0 Jul 03 14:17:09.181629 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.181237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerDied","Data":"c988bcb03bd69b3c67b21ba2a40e4a78f3f2d3e1d53e3c23489c88f0e92f9b67"} Jul 03 14:17:09.183317 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.183298 2580 generic.go:356] "Generic (PLEG): container finished" podID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerID="2ca21717b728db529df888b38c210d63ac58f24eaf16b18cad41e273ac9b7703" exitCode=0 Jul 03 14:17:09.183405 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.183381 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" event={"ID":"56eabe35-4d42-43f3-9597-3688a32b8f62","Type":"ContainerDied","Data":"2ca21717b728db529df888b38c210d63ac58f24eaf16b18cad41e273ac9b7703"} Jul 03 14:17:09.185395 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.185370 2580 generic.go:356] "Generic (PLEG): container finished" podID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerID="426961bb0bba93ab15bdc5b50be26127f0cd21cbca7652810dbcee6ea13dd73a" exitCode=0 Jul 03 14:17:09.185512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.185455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerDied","Data":"426961bb0bba93ab15bdc5b50be26127f0cd21cbca7652810dbcee6ea13dd73a"} Jul 03 14:17:09.187308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.187290 2580 generic.go:356] "Generic (PLEG): container finished" podID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerID="949e641eb0bc2a09db930c1b478e33dac945d8828f80b37e9da0b0d43be92267" exitCode=0 Jul 03 14:17:09.187394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:09.187320 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" event={"ID":"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c","Type":"ContainerDied","Data":"949e641eb0bc2a09db930c1b478e33dac945d8828f80b37e9da0b0d43be92267"} Jul 03 14:17:10.331492 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.331462 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:10.398294 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.398268 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:10.401576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.401547 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:10.404432 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.404415 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:10.425378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.425350 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle\") pod \"56eabe35-4d42-43f3-9597-3688a32b8f62\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " Jul 03 14:17:10.425514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.425423 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util\") pod \"56eabe35-4d42-43f3-9597-3688a32b8f62\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " Jul 03 14:17:10.425514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.425492 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xqtq\" (UniqueName: \"kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq\") pod \"56eabe35-4d42-43f3-9597-3688a32b8f62\" (UID: \"56eabe35-4d42-43f3-9597-3688a32b8f62\") " Jul 03 14:17:10.426481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.426447 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle" (OuterVolumeSpecName: "bundle") pod "56eabe35-4d42-43f3-9597-3688a32b8f62" (UID: "56eabe35-4d42-43f3-9597-3688a32b8f62"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.427779 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.427750 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq" (OuterVolumeSpecName: "kube-api-access-8xqtq") pod "56eabe35-4d42-43f3-9597-3688a32b8f62" (UID: "56eabe35-4d42-43f3-9597-3688a32b8f62"). InnerVolumeSpecName "kube-api-access-8xqtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:10.430968 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.430944 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util" (OuterVolumeSpecName: "util") pod "56eabe35-4d42-43f3-9597-3688a32b8f62" (UID: "56eabe35-4d42-43f3-9597-3688a32b8f62"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.526116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526037 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util\") pod \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " Jul 03 14:17:10.526116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526094 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle\") pod \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util\") pod \"409a5eb6-527d-4915-b6a2-b0a976726af0\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526144 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle\") pod \"409a5eb6-527d-4915-b6a2-b0a976726af0\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526170 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle\") pod \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526198 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf9mn\" (UniqueName: \"kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn\") pod \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526237 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68l7t\" (UniqueName: \"kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t\") pod \"409a5eb6-527d-4915-b6a2-b0a976726af0\" (UID: \"409a5eb6-527d-4915-b6a2-b0a976726af0\") " Jul 03 14:17:10.526319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526267 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zktqs\" (UniqueName: \"kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs\") pod \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\" (UID: \"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c\") " Jul 03 14:17:10.526661 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526395 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util\") pod \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\" (UID: \"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc\") " Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526763 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526788 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56eabe35-4d42-43f3-9597-3688a32b8f62-util\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526804 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xqtq\" (UniqueName: \"kubernetes.io/projected/56eabe35-4d42-43f3-9597-3688a32b8f62-kube-api-access-8xqtq\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526780 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle" (OuterVolumeSpecName: "bundle") pod "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" (UID: "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.526886 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle" (OuterVolumeSpecName: "bundle") pod "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" (UID: "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.527476 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.527425 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle" (OuterVolumeSpecName: "bundle") pod "409a5eb6-527d-4915-b6a2-b0a976726af0" (UID: "409a5eb6-527d-4915-b6a2-b0a976726af0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.528771 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.528746 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs" (OuterVolumeSpecName: "kube-api-access-zktqs") pod "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" (UID: "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c"). InnerVolumeSpecName "kube-api-access-zktqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:10.528867 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.528776 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t" (OuterVolumeSpecName: "kube-api-access-68l7t") pod "409a5eb6-527d-4915-b6a2-b0a976726af0" (UID: "409a5eb6-527d-4915-b6a2-b0a976726af0"). InnerVolumeSpecName "kube-api-access-68l7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:10.528867 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.528790 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn" (OuterVolumeSpecName: "kube-api-access-vf9mn") pod "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" (UID: "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc"). InnerVolumeSpecName "kube-api-access-vf9mn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:10.532564 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.532543 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util" (OuterVolumeSpecName: "util") pod "409a5eb6-527d-4915-b6a2-b0a976726af0" (UID: "409a5eb6-527d-4915-b6a2-b0a976726af0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.532633 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.532592 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util" (OuterVolumeSpecName: "util") pod "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" (UID: "bcc27fa0-cb5b-4f48-abfd-3784460fbc5c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.533154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.533132 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util" (OuterVolumeSpecName: "util") pod "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" (UID: "fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:10.628023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.627994 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628018 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-util\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628033 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/409a5eb6-527d-4915-b6a2-b0a976726af0-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628043 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628051 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf9mn\" (UniqueName: \"kubernetes.io/projected/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-kube-api-access-vf9mn\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628062 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68l7t\" (UniqueName: \"kubernetes.io/projected/409a5eb6-527d-4915-b6a2-b0a976726af0-kube-api-access-68l7t\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628071 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zktqs\" (UniqueName: \"kubernetes.io/projected/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-kube-api-access-zktqs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628080 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc-util\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:10.628271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:10.628088 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcc27fa0-cb5b-4f48-abfd-3784460fbc5c-util\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:11.199169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.199141 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" Jul 03 14:17:11.201049 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.201026 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" Jul 03 14:17:11.203036 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.203009 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" Jul 03 14:17:11.205154 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205135 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8" event={"ID":"bcc27fa0-cb5b-4f48-abfd-3784460fbc5c","Type":"ContainerDied","Data":"d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7"} Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205864 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6feba084646107fa21ed0eaf2216bd1b45b4b65c522680f7aa3ee0fc07388c7" Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205879 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7" event={"ID":"fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc","Type":"ContainerDied","Data":"06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092"} Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205899 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ec9530f232a0af659b65861cfe69ee5bc1618ac975287e8e4cf9cdf9658092" Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b" event={"ID":"56eabe35-4d42-43f3-9597-3688a32b8f62","Type":"ContainerDied","Data":"6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c"} Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205929 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a95321843111c8c10f79dd18c4f884c72e41be02078a53ebc5b460e17c8d10c" Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205941 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4" event={"ID":"409a5eb6-527d-4915-b6a2-b0a976726af0","Type":"ContainerDied","Data":"7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9"} Jul 03 14:17:11.209774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:11.205962 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7368ba00f03a5260ed72a7b4649c54922b6ee30c41c32ecd35e0888aacd9a9e9" Jul 03 14:17:13.099890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.099852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:13.138291 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.138257 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c56bf885c-rjgn9" podUID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" containerName="console" containerID="cri-o://54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc" gracePeriod=15 Jul 03 14:17:13.376290 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.376264 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:17:13.451632 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451598 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.451816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451650 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dz9x\" (UniqueName: \"kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.451816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451681 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.451816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451715 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.451816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451762 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.451816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451791 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.452089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.451832 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert\") pod \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\" (UID: \"d3a38c82-6a27-43af-a4ed-92b9e9f7f316\") " Jul 03 14:17:13.452201 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.452173 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config" (OuterVolumeSpecName: "console-config") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:17:13.452201 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.452183 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca" (OuterVolumeSpecName: "service-ca") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:17:13.452278 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.452252 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:17:13.452474 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.452451 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:17:13.453847 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.453826 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:17:13.453926 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.453873 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:17:13.453974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.453934 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x" (OuterVolumeSpecName: "kube-api-access-4dz9x") pod "d3a38c82-6a27-43af-a4ed-92b9e9f7f316" (UID: "d3a38c82-6a27-43af-a4ed-92b9e9f7f316"). InnerVolumeSpecName "kube-api-access-4dz9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:13.552394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552360 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-oauth-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552385 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dz9x\" (UniqueName: \"kubernetes.io/projected/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-kube-api-access-4dz9x\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552394 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-config\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552405 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-service-ca\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552414 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-console-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552422 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-trusted-ca-bundle\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:13.552637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:13.552430 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3a38c82-6a27-43af-a4ed-92b9e9f7f316-oauth-serving-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:14.219449 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.219414 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" containerID="54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc" exitCode=0 Jul 03 14:17:14.219449 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.219455 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c56bf885c-rjgn9" Jul 03 14:17:14.220000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.219498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c56bf885c-rjgn9" event={"ID":"d3a38c82-6a27-43af-a4ed-92b9e9f7f316","Type":"ContainerDied","Data":"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc"} Jul 03 14:17:14.220000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.219551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c56bf885c-rjgn9" event={"ID":"d3a38c82-6a27-43af-a4ed-92b9e9f7f316","Type":"ContainerDied","Data":"3938c66a50787709f6751c66b9d7bad6363e9060e75478898f17cf6c5c189e17"} Jul 03 14:17:14.220000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.219582 2580 scope.go:117] "RemoveContainer" containerID="54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc" Jul 03 14:17:14.228477 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.228463 2580 scope.go:117] "RemoveContainer" containerID="54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc" Jul 03 14:17:14.228680 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:14.228662 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc\": container with ID starting with 54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc not found: ID does not exist" containerID="54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc" Jul 03 14:17:14.228719 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.228687 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc"} err="failed to get container status \"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc\": rpc error: code = NotFound desc = could not find container \"54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc\": container with ID starting with 54a6844f99c3ca2d6c99eaa76f18d974eb782917f9577087ab5980fa838f8fbc not found: ID does not exist" Jul 03 14:17:14.244443 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.244419 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:17:14.253371 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:14.253346 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c56bf885c-rjgn9"] Jul 03 14:17:15.204325 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:15.204292 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" path="/var/lib/kubelet/pods/d3a38c82-6a27-43af-a4ed-92b9e9f7f316/volumes" Jul 03 14:17:16.907027 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:16.906995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:17.902215 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902183 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2"] Jul 03 14:17:17.902491 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902479 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="extract" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902493 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="extract" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902505 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="util" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902510 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="util" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902515 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="extract" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902521 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="extract" Jul 03 14:17:17.902530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902530 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902535 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902541 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902546 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902552 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" containerName="console" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902557 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" containerName="console" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902562 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902567 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902575 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902580 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902585 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902590 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902599 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902603 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902609 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902613 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="util" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902618 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902623 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902629 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902633 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="pull" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902680 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a38c82-6a27-43af-a4ed-92b9e9f7f316" containerName="console" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902690 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="409a5eb6-527d-4915-b6a2-b0a976726af0" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902698 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902703 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56eabe35-4d42-43f3-9597-3688a32b8f62" containerName="extract" Jul 03 14:17:17.902699 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.902709 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcc27fa0-cb5b-4f48-abfd-3784460fbc5c" containerName="extract" Jul 03 14:17:17.907077 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.907060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:17.922584 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.922563 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kd9nd\"" Jul 03 14:17:17.944581 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.944559 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2"] Jul 03 14:17:17.987376 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.987344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljb6\" (UniqueName: \"kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:17.987499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:17.987402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.088865 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.088830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.089039 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.088897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljb6\" (UniqueName: \"kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.089294 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.089275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.119230 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.119197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljb6\" (UniqueName: \"kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.127411 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.127383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:18.216225 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.216148 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:18.378302 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.378277 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2"] Jul 03 14:17:18.380967 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:18.380936 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbde7732_b324_4e5a_92b6_2e195fadc2bc.slice/crio-dfbe87051e5558c749839a21c739a435d42b3b355a1262d0bdc359ce780376bf WatchSource:0}: Error finding container dfbe87051e5558c749839a21c739a435d42b3b355a1262d0bdc359ce780376bf: Status 404 returned error can't find the container with id dfbe87051e5558c749839a21c739a435d42b3b355a1262d0bdc359ce780376bf Jul 03 14:17:18.579982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:18.579955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:19.247918 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:19.247868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" event={"ID":"cbde7732-b324-4e5a-92b6-2e195fadc2bc","Type":"ContainerStarted","Data":"dfbe87051e5558c749839a21c739a435d42b3b355a1262d0bdc359ce780376bf"} Jul 03 14:17:19.474077 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:19.474046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:20.913666 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:20.913632 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-66r66"] Jul 03 14:17:20.916953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:20.916928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:20.919718 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:20.919698 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wd6hz\"" Jul 03 14:17:20.943732 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:20.943703 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-66r66"] Jul 03 14:17:21.013177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.013140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kwrc\" (UniqueName: \"kubernetes.io/projected/dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7-kube-api-access-5kwrc\") pod \"authorino-operator-75cc8d5cb7-66r66\" (UID: \"dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:21.114326 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.114291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kwrc\" (UniqueName: \"kubernetes.io/projected/dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7-kube-api-access-5kwrc\") pod \"authorino-operator-75cc8d5cb7-66r66\" (UID: \"dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:21.133268 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.133236 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kwrc\" (UniqueName: \"kubernetes.io/projected/dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7-kube-api-access-5kwrc\") pod \"authorino-operator-75cc8d5cb7-66r66\" (UID: \"dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:21.229524 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.229445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:21.399786 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.399757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-66r66"] Jul 03 14:17:21.401020 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:21.400990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbda8472_972e_4ff4_b5a8_8b6c8b9b87d7.slice/crio-2f1a03dcaa0f2c855ac946a52e97bc3a4a05254c2985a1074f0218c6dc0aa163 WatchSource:0}: Error finding container 2f1a03dcaa0f2c855ac946a52e97bc3a4a05254c2985a1074f0218c6dc0aa163: Status 404 returned error can't find the container with id 2f1a03dcaa0f2c855ac946a52e97bc3a4a05254c2985a1074f0218c6dc0aa163 Jul 03 14:17:21.450592 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:21.450559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:22.269523 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.269478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" event={"ID":"dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7","Type":"ContainerStarted","Data":"2f1a03dcaa0f2c855ac946a52e97bc3a4a05254c2985a1074f0218c6dc0aa163"} Jul 03 14:17:22.784547 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.784511 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk"] Jul 03 14:17:22.787970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.787946 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:22.792210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.792186 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 03 14:17:22.792321 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.792215 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dwhm7\"" Jul 03 14:17:22.806670 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.806644 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk"] Jul 03 14:17:22.930139 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:22.930106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4sjn\" (UniqueName: \"kubernetes.io/projected/5cda36ff-3991-4960-b497-1bab3a623bec-kube-api-access-k4sjn\") pod \"dns-operator-controller-manager-864dd78c4c-9jqbk\" (UID: \"5cda36ff-3991-4960-b497-1bab3a623bec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:23.030834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:23.030794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4sjn\" (UniqueName: \"kubernetes.io/projected/5cda36ff-3991-4960-b497-1bab3a623bec-kube-api-access-k4sjn\") pod \"dns-operator-controller-manager-864dd78c4c-9jqbk\" (UID: \"5cda36ff-3991-4960-b497-1bab3a623bec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:23.046603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:23.046540 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4sjn\" (UniqueName: \"kubernetes.io/projected/5cda36ff-3991-4960-b497-1bab3a623bec-kube-api-access-k4sjn\") pod \"dns-operator-controller-manager-864dd78c4c-9jqbk\" (UID: \"5cda36ff-3991-4960-b497-1bab3a623bec\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:23.101102 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:23.101067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:23.355969 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:23.355926 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:23.637890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:23.637809 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:24.907170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:24.907147 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk"] Jul 03 14:17:24.910209 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:24.910187 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cda36ff_3991_4960_b497_1bab3a623bec.slice/crio-5ab2beae01ffd3752b17d09a19c4cb06151bf1639964d7a7328933dcd9635168 WatchSource:0}: Error finding container 5ab2beae01ffd3752b17d09a19c4cb06151bf1639964d7a7328933dcd9635168: Status 404 returned error can't find the container with id 5ab2beae01ffd3752b17d09a19c4cb06151bf1639964d7a7328933dcd9635168 Jul 03 14:17:25.288103 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.288066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" event={"ID":"dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7","Type":"ContainerStarted","Data":"a6d54a0816f963cf75e8ee83a0383ec6799f87e68d4ce0727d615e1b14240eda"} Jul 03 14:17:25.288311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.288132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:25.289300 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.289271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" event={"ID":"5cda36ff-3991-4960-b497-1bab3a623bec","Type":"ContainerStarted","Data":"5ab2beae01ffd3752b17d09a19c4cb06151bf1639964d7a7328933dcd9635168"} Jul 03 14:17:25.290775 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.290755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" event={"ID":"cbde7732-b324-4e5a-92b6-2e195fadc2bc","Type":"ContainerStarted","Data":"ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5"} Jul 03 14:17:25.290887 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.290869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:25.319013 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.318969 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" podStartSLOduration=1.9011558530000001 podStartE2EDuration="5.318953476s" totalImagesPullingTime="3.417797623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:25.31747036 +0000 UTC m=+538.739234835" watchObservedRunningTime="2026-07-03 14:17:25.318953476 +0000 UTC m=+538.740717954" Jul 03 14:17:25.369270 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.369224 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" podStartSLOduration=1.93973329 podStartE2EDuration="8.369210755s" totalImagesPullingTime="6.429477465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:25.366455225 +0000 UTC m=+538.788219716" watchObservedRunningTime="2026-07-03 14:17:25.369210755 +0000 UTC m=+538.790975236" Jul 03 14:17:25.488791 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.488757 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm"] Jul 03 14:17:25.492338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.492315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:25.500158 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.500127 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rmjgl\"" Jul 03 14:17:25.506762 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.506722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm"] Jul 03 14:17:25.653915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.653884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx2qw\" (UniqueName: \"kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw\") pod \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" (UID: \"5faffaa1-b829-402f-bad0-5fac5ce94920\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:25.754993 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.754954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx2qw\" (UniqueName: \"kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw\") pod \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" (UID: \"5faffaa1-b829-402f-bad0-5fac5ce94920\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:25.774745 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.774703 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx2qw\" (UniqueName: \"kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw\") pod \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" (UID: \"5faffaa1-b829-402f-bad0-5fac5ce94920\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:25.802405 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.802360 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:25.979423 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:25.979357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:26.220748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:26.220698 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm"] Jul 03 14:17:26.270395 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:26.270318 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5faffaa1_b829_402f_bad0_5fac5ce94920.slice/crio-4ed6571fe6e115c3d7636c407e1b790abdfd176939714959af8f904369b5f900 WatchSource:0}: Error finding container 4ed6571fe6e115c3d7636c407e1b790abdfd176939714959af8f904369b5f900: Status 404 returned error can't find the container with id 4ed6571fe6e115c3d7636c407e1b790abdfd176939714959af8f904369b5f900 Jul 03 14:17:26.297834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:26.297793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" event={"ID":"5faffaa1-b829-402f-bad0-5fac5ce94920","Type":"ContainerStarted","Data":"4ed6571fe6e115c3d7636c407e1b790abdfd176939714959af8f904369b5f900"} Jul 03 14:17:27.151015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:27.150992 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 03 14:17:27.948894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:27.948859 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:28.049383 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.049354 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:28.136598 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.136573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:28.311226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.311131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" event={"ID":"5faffaa1-b829-402f-bad0-5fac5ce94920","Type":"ContainerStarted","Data":"324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997"} Jul 03 14:17:28.311226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.311200 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:28.312648 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.312625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" event={"ID":"5cda36ff-3991-4960-b497-1bab3a623bec","Type":"ContainerStarted","Data":"7742ceb6c39c540d850fa8590ad0fea25918a179225e907e10ac4162480dbae6"} Jul 03 14:17:28.312793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.312778 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:28.358323 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.358265 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" podStartSLOduration=1.552131112 podStartE2EDuration="3.358242953s" totalImagesPullingTime="1.806111841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:28.355261012 +0000 UTC m=+541.777025493" watchObservedRunningTime="2026-07-03 14:17:28.358242953 +0000 UTC m=+541.780007439" Jul 03 14:17:28.389418 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:28.389373 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" podStartSLOduration=4.153346909 podStartE2EDuration="6.389359319s" totalImagesPullingTime="2.23601241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:28.386107288 +0000 UTC m=+541.807871770" watchObservedRunningTime="2026-07-03 14:17:28.389359319 +0000 UTC m=+541.811123800" Jul 03 14:17:35.780231 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:35.780197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:35.910959 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:35.910918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:36.300367 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:36.300330 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:36.300558 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:36.300417 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-66r66" Jul 03 14:17:37.562761 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:37.562710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:37.713096 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:37.713064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:38.117478 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.117440 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:38.342053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.342005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:38.734970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.734933 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2"] Jul 03 14:17:38.735342 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.735161 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" containerName="manager" containerID="cri-o://ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5" gracePeriod=2 Jul 03 14:17:38.746568 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.746541 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2"] Jul 03 14:17:38.771070 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.771046 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm"] Jul 03 14:17:38.771237 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.771216 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" containerName="manager" containerID="cri-o://324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997" gracePeriod=2 Jul 03 14:17:38.772765 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.772742 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:38.787763 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.787713 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:38.788181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.788160 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm"] Jul 03 14:17:38.790588 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.790549 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:38.792941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.792921 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:38.793368 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793354 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" containerName="manager" Jul 03 14:17:38.793410 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793372 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" containerName="manager" Jul 03 14:17:38.793410 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793389 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" containerName="manager" Jul 03 14:17:38.793410 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793398 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" containerName="manager" Jul 03 14:17:38.793505 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793484 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" containerName="manager" Jul 03 14:17:38.793505 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.793493 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" containerName="manager" Jul 03 14:17:38.796814 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.796795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.844057 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.844018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:38.845923 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.845898 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl"] Jul 03 14:17:38.848834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.848812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:38.850601 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.850572 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:38.865077 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.865055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.865160 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.865093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkfp7\" (UniqueName: \"kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.865160 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.865131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6xkp\" (UniqueName: \"kubernetes.io/projected/16ed0a24-3715-4875-9b94-d6b0f2a76953-kube-api-access-j6xkp\") pod \"limitador-operator-controller-manager-859d7b78bd-58dtl\" (UID: \"16ed0a24-3715-4875-9b94-d6b0f2a76953\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:38.875386 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.875360 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl"] Jul 03 14:17:38.926978 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.926936 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:38.932929 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.932899 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:38.937167 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:38.937135 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:38.965851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.965691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6xkp\" (UniqueName: \"kubernetes.io/projected/16ed0a24-3715-4875-9b94-d6b0f2a76953-kube-api-access-j6xkp\") pod \"limitador-operator-controller-manager-859d7b78bd-58dtl\" (UID: \"16ed0a24-3715-4875-9b94-d6b0f2a76953\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:38.965851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.965780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.965851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.965828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkfp7\" (UniqueName: \"kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.966214 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.966190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:38.998600 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:38.998545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6xkp\" (UniqueName: \"kubernetes.io/projected/16ed0a24-3715-4875-9b94-d6b0f2a76953-kube-api-access-j6xkp\") pod \"limitador-operator-controller-manager-859d7b78bd-58dtl\" (UID: \"16ed0a24-3715-4875-9b94-d6b0f2a76953\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:39.001703 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.001674 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkfp7\" (UniqueName: \"kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wtnn\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:39.021123 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.021104 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:39.023043 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.023027 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:39.024777 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.024754 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:39.031688 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.031665 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:39.033993 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.033971 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vn5v2\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:39.037073 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.037052 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qqgxm\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:39.066885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.066864 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lljb6\" (UniqueName: \"kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6\") pod \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " Jul 03 14:17:39.066973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.066893 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx2qw\" (UniqueName: \"kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw\") pod \"5faffaa1-b829-402f-bad0-5fac5ce94920\" (UID: \"5faffaa1-b829-402f-bad0-5fac5ce94920\") " Jul 03 14:17:39.067188 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.066988 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume\") pod \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\" (UID: \"cbde7732-b324-4e5a-92b6-2e195fadc2bc\") " Jul 03 14:17:39.067348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.067327 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cbde7732-b324-4e5a-92b6-2e195fadc2bc" (UID: "cbde7732-b324-4e5a-92b6-2e195fadc2bc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:39.068848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.068820 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw" (OuterVolumeSpecName: "kube-api-access-rx2qw") pod "5faffaa1-b829-402f-bad0-5fac5ce94920" (UID: "5faffaa1-b829-402f-bad0-5fac5ce94920"). InnerVolumeSpecName "kube-api-access-rx2qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:39.068848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.068840 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6" (OuterVolumeSpecName: "kube-api-access-lljb6") pod "cbde7732-b324-4e5a-92b6-2e195fadc2bc" (UID: "cbde7732-b324-4e5a-92b6-2e195fadc2bc"). InnerVolumeSpecName "kube-api-access-lljb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:39.167945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.167890 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbde7732-b324-4e5a-92b6-2e195fadc2bc-extensions-socket-volume\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:39.167945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.167936 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lljb6\" (UniqueName: \"kubernetes.io/projected/cbde7732-b324-4e5a-92b6-2e195fadc2bc-kube-api-access-lljb6\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:39.167945 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.167952 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx2qw\" (UniqueName: \"kubernetes.io/projected/5faffaa1-b829-402f-bad0-5fac5ce94920-kube-api-access-rx2qw\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:39.200508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.200472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:39.205113 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.205088 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5faffaa1-b829-402f-bad0-5fac5ce94920" path="/var/lib/kubelet/pods/5faffaa1-b829-402f-bad0-5fac5ce94920/volumes" Jul 03 14:17:39.205464 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.205451 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" path="/var/lib/kubelet/pods/cbde7732-b324-4e5a-92b6-2e195fadc2bc/volumes" Jul 03 14:17:39.205878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.205855 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:39.318902 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.318864 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9jqbk" Jul 03 14:17:39.362821 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.362793 2580 generic.go:356] "Generic (PLEG): container finished" podID="5faffaa1-b829-402f-bad0-5fac5ce94920" containerID="324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997" exitCode=0 Jul 03 14:17:39.362987 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.362837 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qqgxm" Jul 03 14:17:39.362987 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.362894 2580 scope.go:117] "RemoveContainer" containerID="324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997" Jul 03 14:17:39.364396 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.364371 2580 generic.go:356] "Generic (PLEG): container finished" podID="cbde7732-b324-4e5a-92b6-2e195fadc2bc" containerID="ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5" exitCode=0 Jul 03 14:17:39.364494 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.364423 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vn5v2" Jul 03 14:17:39.369982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.369959 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:39.372881 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.372858 2580 scope.go:117] "RemoveContainer" containerID="324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997" Jul 03 14:17:39.373241 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.373206 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997\": container with ID starting with 324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997 not found: ID does not exist" containerID="324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997" Jul 03 14:17:39.373348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.373244 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997"} err="failed to get container status \"324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997\": rpc error: code = NotFound desc = could not find container \"324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997\": container with ID starting with 324aaf2476813086bff59ba65fa1057a7a5a5d9c1c9ffe078288d5e37a467997 not found: ID does not exist" Jul 03 14:17:39.373348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.373268 2580 scope.go:117] "RemoveContainer" containerID="ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5" Jul 03 14:17:39.374455 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:39.374429 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3e7e67_2574_4768_9af4_ad339e64d3d2.slice/crio-315a85aa1b0cee30898b6b2e73a3525bfcbb0a2064542715243cd378f750a6c0 WatchSource:0}: Error finding container 315a85aa1b0cee30898b6b2e73a3525bfcbb0a2064542715243cd378f750a6c0: Status 404 returned error can't find the container with id 315a85aa1b0cee30898b6b2e73a3525bfcbb0a2064542715243cd378f750a6c0 Jul 03 14:17:39.381734 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.381705 2580 scope.go:117] "RemoveContainer" containerID="ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5" Jul 03 14:17:39.382011 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:39.381994 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5\": container with ID starting with ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5 not found: ID does not exist" containerID="ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5" Jul 03 14:17:39.382067 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.382015 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5"} err="failed to get container status \"ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5\": rpc error: code = NotFound desc = could not find container \"ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5\": container with ID starting with ca9ec13f4d8f585a372020ba1dbe4f27f5ad50181f17ff80970ae6f04d4a11e5 not found: ID does not exist" Jul 03 14:17:39.402371 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:39.402343 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl"] Jul 03 14:17:39.407855 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:39.407829 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ed0a24_3715_4875_9b94_d6b0f2a76953.slice/crio-e6568abc0762e335f953f790710dcaf2882ba6a9bbce44d0b33b1b1822ff337b WatchSource:0}: Error finding container e6568abc0762e335f953f790710dcaf2882ba6a9bbce44d0b33b1b1822ff337b: Status 404 returned error can't find the container with id e6568abc0762e335f953f790710dcaf2882ba6a9bbce44d0b33b1b1822ff337b Jul 03 14:17:40.370619 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.370584 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" event={"ID":"16ed0a24-3715-4875-9b94-d6b0f2a76953","Type":"ContainerStarted","Data":"57a00397a19ea464b64b03b647d57074f3dbbf9695090703ad57d2dc82ae040b"} Jul 03 14:17:40.371098 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.370643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" event={"ID":"16ed0a24-3715-4875-9b94-d6b0f2a76953","Type":"ContainerStarted","Data":"e6568abc0762e335f953f790710dcaf2882ba6a9bbce44d0b33b1b1822ff337b"} Jul 03 14:17:40.371098 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.370684 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:40.373169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.373146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" event={"ID":"bf3e7e67-2574-4768-9af4-ad339e64d3d2","Type":"ContainerStarted","Data":"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38"} Jul 03 14:17:40.373328 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.373210 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:40.373328 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.373230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" event={"ID":"bf3e7e67-2574-4768-9af4-ad339e64d3d2","Type":"ContainerStarted","Data":"315a85aa1b0cee30898b6b2e73a3525bfcbb0a2064542715243cd378f750a6c0"} Jul 03 14:17:40.486126 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.486071 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" podStartSLOduration=2.486053116 podStartE2EDuration="2.486053116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:40.465961908 +0000 UTC m=+553.887726389" watchObservedRunningTime="2026-07-03 14:17:40.486053116 +0000 UTC m=+553.907817596" Jul 03 14:17:40.541342 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.541290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" podStartSLOduration=2.541273083 podStartE2EDuration="2.541273083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:40.539720098 +0000 UTC m=+553.961484576" watchObservedRunningTime="2026-07-03 14:17:40.541273083 +0000 UTC m=+553.963037564" Jul 03 14:17:40.776087 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.775998 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:17:40.779397 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.779381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.818071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.818027 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:17:40.845878 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.845855 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:40.882896 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.882844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.882896 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.882897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k7fw\" (UniqueName: \"kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.983905 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.983865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.984088 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.983921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k7fw\" (UniqueName: \"kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.984257 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.984238 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:40.997605 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:40.997576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k7fw\" (UniqueName: \"kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw\") pod \"kuadrant-operator-controller-manager-86878c769f-q4zx2\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:41.056630 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.056552 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:41.088679 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.088652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:41.256118 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.256092 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:17:41.258640 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:17:41.258604 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadff8f2a_68aa_4fde_82cd_81d5db73b7e3.slice/crio-a9b149eb79083ad637cc786f87d3e68b23302b46aa178889c8454f543cff004e WatchSource:0}: Error finding container a9b149eb79083ad637cc786f87d3e68b23302b46aa178889c8454f543cff004e: Status 404 returned error can't find the container with id a9b149eb79083ad637cc786f87d3e68b23302b46aa178889c8454f543cff004e Jul 03 14:17:41.381265 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.381226 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" event={"ID":"adff8f2a-68aa-4fde-82cd-81d5db73b7e3","Type":"ContainerStarted","Data":"73d387ded50f3a67a571310744e4221eab5890a311ac38f4e9635b4a6b57a181"} Jul 03 14:17:41.381666 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.381612 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:41.381666 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.381655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" event={"ID":"adff8f2a-68aa-4fde-82cd-81d5db73b7e3","Type":"ContainerStarted","Data":"a9b149eb79083ad637cc786f87d3e68b23302b46aa178889c8454f543cff004e"} Jul 03 14:17:41.406465 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:41.406397 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" podStartSLOduration=1.406375207 podStartE2EDuration="1.406375207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:41.404691283 +0000 UTC m=+554.826455765" watchObservedRunningTime="2026-07-03 14:17:41.406375207 +0000 UTC m=+554.828139691" Jul 03 14:17:43.966964 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:43.966933 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:44.073927 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:44.073891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:45.152860 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:45.152828 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:45.264189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:45.264163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:48.997908 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:48.997861 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:49.207704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:49.207675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:51.383304 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:51.383274 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-58dtl" Jul 03 14:17:51.383694 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:51.383423 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:53.392769 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.392740 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:17:53.472964 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.472936 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:53.473135 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.473112 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" podUID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" containerName="manager" containerID="cri-o://ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38" gracePeriod=10 Jul 03 14:17:53.721045 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.721019 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:53.782507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.782473 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume\") pod \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " Jul 03 14:17:53.782694 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.782525 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkfp7\" (UniqueName: \"kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7\") pod \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\" (UID: \"bf3e7e67-2574-4768-9af4-ad339e64d3d2\") " Jul 03 14:17:53.782874 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.782849 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf3e7e67-2574-4768-9af4-ad339e64d3d2" (UID: "bf3e7e67-2574-4768-9af4-ad339e64d3d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:17:53.784525 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.784503 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7" (OuterVolumeSpecName: "kube-api-access-kkfp7") pod "bf3e7e67-2574-4768-9af4-ad339e64d3d2" (UID: "bf3e7e67-2574-4768-9af4-ad339e64d3d2"). InnerVolumeSpecName "kube-api-access-kkfp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:17:53.882990 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.882957 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf3e7e67-2574-4768-9af4-ad339e64d3d2-extensions-socket-volume\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:53.882990 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:53.882985 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkfp7\" (UniqueName: \"kubernetes.io/projected/bf3e7e67-2574-4768-9af4-ad339e64d3d2-kube-api-access-kkfp7\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:17:54.443721 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.443686 2580 generic.go:356] "Generic (PLEG): container finished" podID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" containerID="ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38" exitCode=0 Jul 03 14:17:54.444288 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.443769 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" Jul 03 14:17:54.444288 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.443786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" event={"ID":"bf3e7e67-2574-4768-9af4-ad339e64d3d2","Type":"ContainerDied","Data":"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38"} Jul 03 14:17:54.444288 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.443843 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn" event={"ID":"bf3e7e67-2574-4768-9af4-ad339e64d3d2","Type":"ContainerDied","Data":"315a85aa1b0cee30898b6b2e73a3525bfcbb0a2064542715243cd378f750a6c0"} Jul 03 14:17:54.444288 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.443865 2580 scope.go:117] "RemoveContainer" containerID="ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38" Jul 03 14:17:54.453359 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.453343 2580 scope.go:117] "RemoveContainer" containerID="ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38" Jul 03 14:17:54.453592 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:17:54.453573 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38\": container with ID starting with ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38 not found: ID does not exist" containerID="ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38" Jul 03 14:17:54.453648 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.453604 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38"} err="failed to get container status \"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38\": rpc error: code = NotFound desc = could not find container \"ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38\": container with ID starting with ead0d8423ae23ae1c0fc89ec826382dc8fa903717f841e06350f15f381b99a38 not found: ID does not exist" Jul 03 14:17:54.481934 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.481906 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:54.487033 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:54.487012 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wtnn"] Jul 03 14:17:55.204876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:55.204847 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" path="/var/lib/kubelet/pods/bf3e7e67-2574-4768-9af4-ad339e64d3d2/volumes" Jul 03 14:17:57.833557 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:57.833513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:17:57.990849 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:17:57.990806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:00.298481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:00.298446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:00.431561 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:00.431534 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:05.076942 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:05.076912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:05.200665 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:05.200628 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:09.055370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:09.055336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:09.282360 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:09.282330 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:10.230451 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:10.230418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:10.508700 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:10.508617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:13.340532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:13.340497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:13.473889 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:13.473860 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:15.056658 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.056622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:15.074221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.074196 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:15.074533 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.074521 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" containerName="manager" Jul 03 14:18:15.074585 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.074534 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" containerName="manager" Jul 03 14:18:15.074622 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.074604 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf3e7e67-2574-4768-9af4-ad339e64d3d2" containerName="manager" Jul 03 14:18:15.079841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.079821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.082869 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.082850 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 03 14:18:15.083144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.083127 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4p5v9\"" Jul 03 14:18:15.088697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.088672 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:15.153150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.153119 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:15.165978 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.165950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84tj5\" (UniqueName: \"kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.166150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.165995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.185114 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.185084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:15.267264 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.267230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84tj5\" (UniqueName: \"kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.267432 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.267286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.267983 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.267963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.286350 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.286321 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84tj5\" (UniqueName: \"kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5\") pod \"limitador-limitador-64c94fd64f-w6m8r\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.323768 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.323745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:15.391067 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.391037 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:15.544612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.544587 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:15.546003 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:18:15.545955 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7462726f_e3ad_4c67_8413_2b8fb361cf4a.slice/crio-76f58d47871ab6c83d6f483b88eeed913cb80032faa6ad234ed18eb4562158c0 WatchSource:0}: Error finding container 76f58d47871ab6c83d6f483b88eeed913cb80032faa6ad234ed18eb4562158c0: Status 404 returned error can't find the container with id 76f58d47871ab6c83d6f483b88eeed913cb80032faa6ad234ed18eb4562158c0 Jul 03 14:18:15.553772 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:15.553715 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" event={"ID":"7462726f-e3ad-4c67-8413-2b8fb361cf4a","Type":"ContainerStarted","Data":"76f58d47871ab6c83d6f483b88eeed913cb80032faa6ad234ed18eb4562158c0"} Jul 03 14:18:16.088271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.088237 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:16.090975 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.090955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:16.094324 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.094303 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-28xd8\"" Jul 03 14:18:16.102944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.102915 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:16.147911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.147879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:16.174898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.174872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl9dj\" (UniqueName: \"kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj\") pod \"authorino-58cd955675-2jsg2\" (UID: \"3e80d201-df48-42ed-90db-66a935e4f2c5\") " pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:16.248921 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.248885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:16.276141 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.276107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dj\" (UniqueName: \"kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj\") pod \"authorino-58cd955675-2jsg2\" (UID: \"3e80d201-df48-42ed-90db-66a935e4f2c5\") " pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:16.326655 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.326622 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl9dj\" (UniqueName: \"kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj\") pod \"authorino-58cd955675-2jsg2\" (UID: \"3e80d201-df48-42ed-90db-66a935e4f2c5\") " pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:16.400816 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.400720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:16.575405 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:16.575273 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:17.570132 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:17.570097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2jsg2" event={"ID":"3e80d201-df48-42ed-90db-66a935e4f2c5","Type":"ContainerStarted","Data":"6559e7b83510fcb8c326d95098d3bd214209e483a204845448dfa82daf89e0cb"} Jul 03 14:18:18.277925 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:18.277891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:18.391743 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:18.391692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:18.577846 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:18.577816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" event={"ID":"7462726f-e3ad-4c67-8413-2b8fb361cf4a","Type":"ContainerStarted","Data":"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f"} Jul 03 14:18:18.578292 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:18.577918 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:18.623432 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:18.623377 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" podStartSLOduration=0.807929778 podStartE2EDuration="3.623358931s" totalImagesPullingTime="2.815429153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:18.618924211 +0000 UTC m=+592.040688694" watchObservedRunningTime="2026-07-03 14:18:18.623358931 +0000 UTC m=+592.045123414" Jul 03 14:18:20.708378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:20.708336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:20.847950 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:20.847914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:21.595888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:21.595847 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2jsg2" event={"ID":"3e80d201-df48-42ed-90db-66a935e4f2c5","Type":"ContainerStarted","Data":"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612"} Jul 03 14:18:21.623340 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:21.623275 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2jsg2" podStartSLOduration=1.2672297430000001 podStartE2EDuration="5.623260081s" totalImagesPullingTime="4.356030338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:21.621704453 +0000 UTC m=+595.043468951" watchObservedRunningTime="2026-07-03 14:18:21.623260081 +0000 UTC m=+595.045024573" Jul 03 14:18:22.873875 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:22.873842 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:23.605177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:23.605139 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2jsg2" podUID="3e80d201-df48-42ed-90db-66a935e4f2c5" containerName="authorino" containerID="cri-o://eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612" gracePeriod=30 Jul 03 14:18:23.843835 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:23.843811 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:23.942794 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:23.942701 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl9dj\" (UniqueName: \"kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj\") pod \"3e80d201-df48-42ed-90db-66a935e4f2c5\" (UID: \"3e80d201-df48-42ed-90db-66a935e4f2c5\") " Jul 03 14:18:23.944707 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:23.944681 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj" (OuterVolumeSpecName: "kube-api-access-hl9dj") pod "3e80d201-df48-42ed-90db-66a935e4f2c5" (UID: "3e80d201-df48-42ed-90db-66a935e4f2c5"). InnerVolumeSpecName "kube-api-access-hl9dj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:18:24.043603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.043566 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl9dj\" (UniqueName: \"kubernetes.io/projected/3e80d201-df48-42ed-90db-66a935e4f2c5-kube-api-access-hl9dj\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:18:24.611540 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.611502 2580 generic.go:356] "Generic (PLEG): container finished" podID="3e80d201-df48-42ed-90db-66a935e4f2c5" containerID="eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612" exitCode=0 Jul 03 14:18:24.611746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.611553 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2jsg2" Jul 03 14:18:24.611746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.611570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2jsg2" event={"ID":"3e80d201-df48-42ed-90db-66a935e4f2c5","Type":"ContainerDied","Data":"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612"} Jul 03 14:18:24.611746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.611608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2jsg2" event={"ID":"3e80d201-df48-42ed-90db-66a935e4f2c5","Type":"ContainerDied","Data":"6559e7b83510fcb8c326d95098d3bd214209e483a204845448dfa82daf89e0cb"} Jul 03 14:18:24.611746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.611632 2580 scope.go:117] "RemoveContainer" containerID="eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612" Jul 03 14:18:24.620318 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.620301 2580 scope.go:117] "RemoveContainer" containerID="eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612" Jul 03 14:18:24.620551 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:18:24.620535 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612\": container with ID starting with eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612 not found: ID does not exist" containerID="eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612" Jul 03 14:18:24.620594 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.620558 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612"} err="failed to get container status \"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612\": rpc error: code = NotFound desc = could not find container \"eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612\": container with ID starting with eed78cff6d2f9bbf69eac9c39b56033f50280a149f0eb0ee58bb49fe31762612 not found: ID does not exist" Jul 03 14:18:24.635069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.635044 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:24.644387 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:24.644361 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2jsg2"] Jul 03 14:18:25.204927 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:25.204894 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e80d201-df48-42ed-90db-66a935e4f2c5" path="/var/lib/kubelet/pods/3e80d201-df48-42ed-90db-66a935e4f2c5/volumes" Jul 03 14:18:25.474481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:25.474402 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:25.847869 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:25.847841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:29.585138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:29.585103 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:29.876627 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:29.876549 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:29.876812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:29.876751 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" podUID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" containerName="limitador" containerID="cri-o://ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f" gracePeriod=30 Jul 03 14:18:30.416024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.415994 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:30.491333 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.491257 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file\") pod \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " Jul 03 14:18:30.491333 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.491293 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84tj5\" (UniqueName: \"kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5\") pod \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\" (UID: \"7462726f-e3ad-4c67-8413-2b8fb361cf4a\") " Jul 03 14:18:30.491651 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.491623 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file" (OuterVolumeSpecName: "config-file") pod "7462726f-e3ad-4c67-8413-2b8fb361cf4a" (UID: "7462726f-e3ad-4c67-8413-2b8fb361cf4a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:18:30.493241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.493220 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5" (OuterVolumeSpecName: "kube-api-access-84tj5") pod "7462726f-e3ad-4c67-8413-2b8fb361cf4a" (UID: "7462726f-e3ad-4c67-8413-2b8fb361cf4a"). InnerVolumeSpecName "kube-api-access-84tj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:18:30.592195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.592163 2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7462726f-e3ad-4c67-8413-2b8fb361cf4a-config-file\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:18:30.592195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.592191 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84tj5\" (UniqueName: \"kubernetes.io/projected/7462726f-e3ad-4c67-8413-2b8fb361cf4a-kube-api-access-84tj5\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:18:30.642558 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.642520 2580 generic.go:356] "Generic (PLEG): container finished" podID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" containerID="ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f" exitCode=0 Jul 03 14:18:30.642759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.642565 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" Jul 03 14:18:30.642759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.642608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" event={"ID":"7462726f-e3ad-4c67-8413-2b8fb361cf4a","Type":"ContainerDied","Data":"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f"} Jul 03 14:18:30.642759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.642667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w6m8r" event={"ID":"7462726f-e3ad-4c67-8413-2b8fb361cf4a","Type":"ContainerDied","Data":"76f58d47871ab6c83d6f483b88eeed913cb80032faa6ad234ed18eb4562158c0"} Jul 03 14:18:30.642759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.642690 2580 scope.go:117] "RemoveContainer" containerID="ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f" Jul 03 14:18:30.651492 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.651463 2580 scope.go:117] "RemoveContainer" containerID="ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f" Jul 03 14:18:30.651775 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:18:30.651754 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f\": container with ID starting with ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f not found: ID does not exist" containerID="ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f" Jul 03 14:18:30.651820 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.651786 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f"} err="failed to get container status \"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f\": rpc error: code = NotFound desc = could not find container \"ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f\": container with ID starting with ff2985844b4e50accbd614152c34fd73fd01e075311d99ffd88053742ee8950f not found: ID does not exist" Jul 03 14:18:30.665308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.665281 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:30.674874 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:30.672971 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w6m8r"] Jul 03 14:18:31.205803 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.205771 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" path="/var/lib/kubelet/pods/7462726f-e3ad-4c67-8413-2b8fb361cf4a/volumes" Jul 03 14:18:31.390305 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390271 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-rnhb4"] Jul 03 14:18:31.390635 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390622 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e80d201-df48-42ed-90db-66a935e4f2c5" containerName="authorino" Jul 03 14:18:31.390680 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390638 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e80d201-df48-42ed-90db-66a935e4f2c5" containerName="authorino" Jul 03 14:18:31.390680 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390648 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" containerName="limitador" Jul 03 14:18:31.390680 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390654 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" containerName="limitador" Jul 03 14:18:31.390793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390751 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e80d201-df48-42ed-90db-66a935e4f2c5" containerName="authorino" Jul 03 14:18:31.390793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.390763 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7462726f-e3ad-4c67-8413-2b8fb361cf4a" containerName="limitador" Jul 03 14:18:31.393360 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.393338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.397286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.397268 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-l8qw4\"" Jul 03 14:18:31.397389 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.397358 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 14:18:31.398437 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.398420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-data\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.398504 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.398454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqbr2\" (UniqueName: \"kubernetes.io/projected/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-kube-api-access-rqbr2\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.407171 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.407143 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rnhb4"] Jul 03 14:18:31.445061 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.445028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:31.498919 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.498828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-data\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.498919 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.498881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqbr2\" (UniqueName: \"kubernetes.io/projected/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-kube-api-access-rqbr2\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.499229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.499208 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-data\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.511574 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.511549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqbr2\" (UniqueName: \"kubernetes.io/projected/f9d29f20-cb42-4b35-b8e1-b21934d8d0dd-kube-api-access-rqbr2\") pod \"postgres-69fb4594cf-rnhb4\" (UID: \"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd\") " pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.703576 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.703536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:31.842614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:31.842581 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-rnhb4"] Jul 03 14:18:31.845539 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:18:31.845513 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d29f20_cb42_4b35_b8e1_b21934d8d0dd.slice/crio-f664f3d7048319a28da7b8ba1a31134c664d85863210ec6dfde29bcf43d73004 WatchSource:0}: Error finding container f664f3d7048319a28da7b8ba1a31134c664d85863210ec6dfde29bcf43d73004: Status 404 returned error can't find the container with id f664f3d7048319a28da7b8ba1a31134c664d85863210ec6dfde29bcf43d73004 Jul 03 14:18:32.654972 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:32.654933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rnhb4" event={"ID":"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd","Type":"ContainerStarted","Data":"f664f3d7048319a28da7b8ba1a31134c664d85863210ec6dfde29bcf43d73004"} Jul 03 14:18:37.689973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:37.689880 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-rnhb4" event={"ID":"f9d29f20-cb42-4b35-b8e1-b21934d8d0dd","Type":"ContainerStarted","Data":"7b6a4cbc1a58f6a38605ced9ff23553adebcfddbf3529f9941ed8d7c555b4771"} Jul 03 14:18:37.690329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:37.690060 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:43.724402 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:43.724373 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-rnhb4" Jul 03 14:18:43.753832 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:43.753788 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-rnhb4" podStartSLOduration=7.273809768 podStartE2EDuration="12.753772751s" totalImagesPullingTime="5.479962983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:37.730429425 +0000 UTC m=+611.152193907" watchObservedRunningTime="2026-07-03 14:18:43.753772751 +0000 UTC m=+617.175537232" Jul 03 14:18:44.075900 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.075868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:44.327384 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.327304 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:44.330929 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.330901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:44.334284 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.334263 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-28xd8\"" Jul 03 14:18:44.348222 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.348196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:44.386599 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.386578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:44.397834 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.397809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj7hf\" (UniqueName: \"kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf\") pod \"authorino-5d4b67c446-76gcc\" (UID: \"dfd19964-6f83-41d7-8747-efee0d838267\") " pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:44.498356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.498323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj7hf\" (UniqueName: \"kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf\") pod \"authorino-5d4b67c446-76gcc\" (UID: \"dfd19964-6f83-41d7-8747-efee0d838267\") " pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:44.513668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.513647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj7hf\" (UniqueName: \"kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf\") pod \"authorino-5d4b67c446-76gcc\" (UID: \"dfd19964-6f83-41d7-8747-efee0d838267\") " pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:44.575671 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.575635 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:44.575915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.575899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:44.619868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.619839 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8d9cb84cc-pbvx8"] Jul 03 14:18:44.624248 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.624222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:44.635432 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.635358 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8d9cb84cc-pbvx8"] Jul 03 14:18:44.700520 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.700483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjzzr\" (UniqueName: \"kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr\") pod \"authorino-8d9cb84cc-pbvx8\" (UID: \"3186d949-f453-4464-953d-45ea83443242\") " pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:44.740925 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.740883 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:44.743468 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:18:44.743441 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfd19964_6f83_41d7_8747_efee0d838267.slice/crio-cf7910bc834a31378e5dc4b1af4d1f380046d95a03b0151844ea9c15b8ad1bf4 WatchSource:0}: Error finding container cf7910bc834a31378e5dc4b1af4d1f380046d95a03b0151844ea9c15b8ad1bf4: Status 404 returned error can't find the container with id cf7910bc834a31378e5dc4b1af4d1f380046d95a03b0151844ea9c15b8ad1bf4 Jul 03 14:18:44.768665 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.768643 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8d9cb84cc-pbvx8"] Jul 03 14:18:44.768896 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:18:44.768876 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xjzzr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" podUID="3186d949-f453-4464-953d-45ea83443242" Jul 03 14:18:44.801098 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.801072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjzzr\" (UniqueName: \"kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr\") pod \"authorino-8d9cb84cc-pbvx8\" (UID: \"3186d949-f453-4464-953d-45ea83443242\") " pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:44.804447 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.804423 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:18:44.814596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.814575 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjzzr\" (UniqueName: \"kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr\") pod \"authorino-8d9cb84cc-pbvx8\" (UID: \"3186d949-f453-4464-953d-45ea83443242\") " pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:44.821570 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.821544 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:18:44.821669 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.821633 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:44.826002 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.825977 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 03 14:18:44.902032 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.901960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp6j9\" (UniqueName: \"kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:44.902032 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:44.902004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.003187 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.003155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp6j9\" (UniqueName: \"kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.003350 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.003201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.005700 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.005675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.015072 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.015049 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp6j9\" (UniqueName: \"kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9\") pod \"authorino-54f496569c-h2tcc\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.131891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.131863 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:18:45.284357 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.284332 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:18:45.286948 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:18:45.286920 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc19c3b6f_1618_4e3b_b387_4f13ebcd0140.slice/crio-80e825272a89e715208b820aa0aa6dfa3401d2892227c00fdd8d3afa66bfc0ae WatchSource:0}: Error finding container 80e825272a89e715208b820aa0aa6dfa3401d2892227c00fdd8d3afa66bfc0ae: Status 404 returned error can't find the container with id 80e825272a89e715208b820aa0aa6dfa3401d2892227c00fdd8d3afa66bfc0ae Jul 03 14:18:45.734511 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.734475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f496569c-h2tcc" event={"ID":"c19c3b6f-1618-4e3b-b387-4f13ebcd0140","Type":"ContainerStarted","Data":"80e825272a89e715208b820aa0aa6dfa3401d2892227c00fdd8d3afa66bfc0ae"} Jul 03 14:18:45.735947 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.735925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:45.735947 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.735937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gcc" event={"ID":"dfd19964-6f83-41d7-8747-efee0d838267","Type":"ContainerStarted","Data":"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4"} Jul 03 14:18:45.736144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.735967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gcc" event={"ID":"dfd19964-6f83-41d7-8747-efee0d838267","Type":"ContainerStarted","Data":"cf7910bc834a31378e5dc4b1af4d1f380046d95a03b0151844ea9c15b8ad1bf4"} Jul 03 14:18:45.736144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.736001 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-76gcc" podUID="dfd19964-6f83-41d7-8747-efee0d838267" containerName="authorino" containerID="cri-o://58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4" gracePeriod=30 Jul 03 14:18:45.754282 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.754238 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-76gcc" podStartSLOduration=1.201934097 podStartE2EDuration="1.754221806s" totalImagesPullingTime="552.287709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:45.754027493 +0000 UTC m=+619.175791975" watchObservedRunningTime="2026-07-03 14:18:45.754221806 +0000 UTC m=+619.175986287" Jul 03 14:18:45.755582 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.755525 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:45.810479 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.810451 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjzzr\" (UniqueName: \"kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr\") pod \"3186d949-f453-4464-953d-45ea83443242\" (UID: \"3186d949-f453-4464-953d-45ea83443242\") " Jul 03 14:18:45.812442 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.812417 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr" (OuterVolumeSpecName: "kube-api-access-xjzzr") pod "3186d949-f453-4464-953d-45ea83443242" (UID: "3186d949-f453-4464-953d-45ea83443242"). InnerVolumeSpecName "kube-api-access-xjzzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:18:45.911212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:45.911186 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjzzr\" (UniqueName: \"kubernetes.io/projected/3186d949-f453-4464-953d-45ea83443242-kube-api-access-xjzzr\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:18:46.024331 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.024301 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:46.113292 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.113249 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj7hf\" (UniqueName: \"kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf\") pod \"dfd19964-6f83-41d7-8747-efee0d838267\" (UID: \"dfd19964-6f83-41d7-8747-efee0d838267\") " Jul 03 14:18:46.115331 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.115292 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf" (OuterVolumeSpecName: "kube-api-access-lj7hf") pod "dfd19964-6f83-41d7-8747-efee0d838267" (UID: "dfd19964-6f83-41d7-8747-efee0d838267"). InnerVolumeSpecName "kube-api-access-lj7hf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:18:46.214458 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.214418 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj7hf\" (UniqueName: \"kubernetes.io/projected/dfd19964-6f83-41d7-8747-efee0d838267-kube-api-access-lj7hf\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:18:46.742319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.742285 2580 generic.go:356] "Generic (PLEG): container finished" podID="dfd19964-6f83-41d7-8747-efee0d838267" containerID="58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4" exitCode=0 Jul 03 14:18:46.742508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.742328 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gcc" Jul 03 14:18:46.742508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.742370 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gcc" event={"ID":"dfd19964-6f83-41d7-8747-efee0d838267","Type":"ContainerDied","Data":"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4"} Jul 03 14:18:46.742508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.742410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gcc" event={"ID":"dfd19964-6f83-41d7-8747-efee0d838267","Type":"ContainerDied","Data":"cf7910bc834a31378e5dc4b1af4d1f380046d95a03b0151844ea9c15b8ad1bf4"} Jul 03 14:18:46.742508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.742431 2580 scope.go:117] "RemoveContainer" containerID="58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4" Jul 03 14:18:46.743956 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.743939 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d9cb84cc-pbvx8" Jul 03 14:18:46.744088 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.743971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f496569c-h2tcc" event={"ID":"c19c3b6f-1618-4e3b-b387-4f13ebcd0140","Type":"ContainerStarted","Data":"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367"} Jul 03 14:18:46.752099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.752079 2580 scope.go:117] "RemoveContainer" containerID="58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4" Jul 03 14:18:46.752422 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:18:46.752401 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4\": container with ID starting with 58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4 not found: ID does not exist" containerID="58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4" Jul 03 14:18:46.752494 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.752430 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4"} err="failed to get container status \"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4\": rpc error: code = NotFound desc = could not find container \"58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4\": container with ID starting with 58918afdb6ec46649537c9f3311ed152a17bd71da6b3bce4adbaac59d267a8c4 not found: ID does not exist" Jul 03 14:18:46.768058 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.768016 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54f496569c-h2tcc" podStartSLOduration=2.310221069 podStartE2EDuration="2.767996166s" totalImagesPullingTime="457.775097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:46.766770543 +0000 UTC m=+620.188535042" watchObservedRunningTime="2026-07-03 14:18:46.767996166 +0000 UTC m=+620.189760647" Jul 03 14:18:46.799980 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.799912 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:46.809067 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.809039 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gcc"] Jul 03 14:18:46.851369 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.851335 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8d9cb84cc-pbvx8"] Jul 03 14:18:46.853676 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:46.853650 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8d9cb84cc-pbvx8"] Jul 03 14:18:47.206429 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:47.206391 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3186d949-f453-4464-953d-45ea83443242" path="/var/lib/kubelet/pods/3186d949-f453-4464-953d-45ea83443242/volumes" Jul 03 14:18:47.206767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:47.206751 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfd19964-6f83-41d7-8747-efee0d838267" path="/var/lib/kubelet/pods/dfd19964-6f83-41d7-8747-efee0d838267/volumes" Jul 03 14:18:51.201708 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.201673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:51.414787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.414751 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:18:51.415101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.415089 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd19964-6f83-41d7-8747-efee0d838267" containerName="authorino" Jul 03 14:18:51.415145 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.415103 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd19964-6f83-41d7-8747-efee0d838267" containerName="authorino" Jul 03 14:18:51.415188 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.415178 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd19964-6f83-41d7-8747-efee0d838267" containerName="authorino" Jul 03 14:18:51.417230 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.417214 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.422970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.422948 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 14:18:51.423087 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.422972 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 14:18:51.424108 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.424062 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dlc6d\"" Jul 03 14:18:51.434156 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.434130 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:18:51.454368 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.454300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.454368 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.454348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5mj\" (UniqueName: \"kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.554858 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.554823 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.555057 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.554875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5mj\" (UniqueName: \"kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.557284 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.557260 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.564229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.564197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5mj\" (UniqueName: \"kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj\") pod \"maas-controller-7bfc956fc6-9lczt\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.727493 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.727394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:51.858596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:51.858572 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:18:52.555190 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:52.555150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:52.778614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:52.778578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" event={"ID":"6846ac31-29b7-41df-808c-458439aea7f6","Type":"ContainerStarted","Data":"27c5e1d41f1cfb57369211bec4efc522bfc9d47df3e22649d50c14e254cc7e23"} Jul 03 14:18:53.785330 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:53.785296 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" event={"ID":"6846ac31-29b7-41df-808c-458439aea7f6","Type":"ContainerStarted","Data":"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4"} Jul 03 14:18:53.785774 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:53.785403 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:18:53.828702 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:53.828641 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" podStartSLOduration=1.06423657 podStartE2EDuration="2.828624707s" totalImagesPullingTime="1.764388137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:53.826541826 +0000 UTC m=+627.248306309" watchObservedRunningTime="2026-07-03 14:18:53.828624707 +0000 UTC m=+627.250389188" Jul 03 14:18:53.843008 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:53.842963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:18:53.965782 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:18:53.965659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:04.795785 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:04.795717 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:19:25.269184 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:25.269152 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:25.791805 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:25.791774 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:26.184520 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.184487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:26.544996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.544914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:26.675784 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.675748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:26.815587 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.815504 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6rx5j"] Jul 03 14:19:26.818782 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.818750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:26.821880 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.821855 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 14:19:26.822043 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.822027 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4n848\"" Jul 03 14:19:26.857190 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.857163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6rx5j"] Jul 03 14:19:26.959939 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.959905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4splc\" (UniqueName: \"kubernetes.io/projected/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-kube-api-access-4splc\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:26.960108 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:26.959955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.060768 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.060715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4splc\" (UniqueName: \"kubernetes.io/projected/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-kube-api-access-4splc\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.060949 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.060792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.066225 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.066161 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 14:19:27.071812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.071788 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4splc\" (UniqueName: \"kubernetes.io/projected/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-kube-api-access-4splc\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.071986 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.071965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05dd2c91-973d-43bb-b48a-4ac051a6dbd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6rx5j\" (UID: \"05dd2c91-973d-43bb-b48a-4ac051a6dbd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.132303 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.132275 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4n848\"" Jul 03 14:19:27.139654 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.139625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:27.279543 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.279518 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6rx5j"] Jul 03 14:19:27.281104 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:19:27.281073 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dd2c91_973d_43bb_b48a_4ac051a6dbd2.slice/crio-21673ab9d0f1acf6993d66a06d1358c9ba585d0f0ff3954f7040a1dfa2a74475 WatchSource:0}: Error finding container 21673ab9d0f1acf6993d66a06d1358c9ba585d0f0ff3954f7040a1dfa2a74475: Status 404 returned error can't find the container with id 21673ab9d0f1acf6993d66a06d1358c9ba585d0f0ff3954f7040a1dfa2a74475 Jul 03 14:19:27.961495 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:27.961460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" event={"ID":"05dd2c91-973d-43bb-b48a-4ac051a6dbd2","Type":"ContainerStarted","Data":"21673ab9d0f1acf6993d66a06d1358c9ba585d0f0ff3954f7040a1dfa2a74475"} Jul 03 14:19:28.968236 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:28.968136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" event={"ID":"05dd2c91-973d-43bb-b48a-4ac051a6dbd2","Type":"ContainerStarted","Data":"95be947924a2edc2953c147469a5ac8e077e4b9def2cb4eb3a64d7bcb3ff6d9e"} Jul 03 14:19:28.968236 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:28.968223 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:28.988000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:28.987942 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" podStartSLOduration=1.617812227 podStartE2EDuration="2.987923916s" totalImagesPullingTime="1.370111689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:19:28.987441657 +0000 UTC m=+662.409206136" watchObservedRunningTime="2026-07-03 14:19:28.987923916 +0000 UTC m=+662.409688399" Jul 03 14:19:36.426394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:36.426365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:36.523931 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:36.523899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:39.974746 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:39.974690 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-6rx5j" Jul 03 14:19:41.354621 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.354584 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f"] Jul 03 14:19:41.357135 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.357114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.360466 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.360444 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xhx89\"" Jul 03 14:19:41.360581 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.360485 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 14:19:41.360777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.360764 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 14:19:41.360827 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.360788 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 03 14:19:41.372874 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.372851 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f"] Jul 03 14:19:41.376814 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.376918 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9kst\" (UniqueName: \"kubernetes.io/projected/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-kube-api-access-t9kst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.376918 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.376918 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.377069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.377069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.376957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.391490 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.391463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:41.477805 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.477769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.477991 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.477881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9kst\" (UniqueName: \"kubernetes.io/projected/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-kube-api-access-t9kst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.477991 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.477918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.477991 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.477944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.477991 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.477966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.478203 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.478023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.478255 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.478228 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.478396 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.478372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.478467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.478424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.480161 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.480145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.480338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.480319 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.487827 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.487804 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9kst\" (UniqueName: \"kubernetes.io/projected/ed6a1d0f-61a1-4a8b-8965-db88b54bef07-kube-api-access-t9kst\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f\" (UID: \"ed6a1d0f-61a1-4a8b-8965-db88b54bef07\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.667637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.667551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:41.801722 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:41.801698 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f"] Jul 03 14:19:41.803676 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:19:41.803645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6a1d0f_61a1_4a8b_8965_db88b54bef07.slice/crio-23c6a9b754e2762592fa56dbf04e823d5a022fd39488ce97c5cf9793dc48081d WatchSource:0}: Error finding container 23c6a9b754e2762592fa56dbf04e823d5a022fd39488ce97c5cf9793dc48081d: Status 404 returned error can't find the container with id 23c6a9b754e2762592fa56dbf04e823d5a022fd39488ce97c5cf9793dc48081d Jul 03 14:19:42.035436 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:42.035356 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" event={"ID":"ed6a1d0f-61a1-4a8b-8965-db88b54bef07","Type":"ContainerStarted","Data":"23c6a9b754e2762592fa56dbf04e823d5a022fd39488ce97c5cf9793dc48081d"} Jul 03 14:19:44.046214 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:44.046178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" event={"ID":"ed6a1d0f-61a1-4a8b-8965-db88b54bef07","Type":"ContainerStarted","Data":"9ac1b8b8419fcbb2bc2ae16f47a5fda43b8811060c9c5d3b5c187123c3505f97"} Jul 03 14:19:44.067948 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:44.067896 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" podStartSLOduration=1.858705074 podStartE2EDuration="3.06787922s" totalImagesPullingTime="1.209174146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:19:44.066985922 +0000 UTC m=+677.488750414" watchObservedRunningTime="2026-07-03 14:19:44.06787922 +0000 UTC m=+677.489643762" Jul 03 14:19:49.171433 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:49.171381 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:19:51.667765 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:51.667713 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:51.668156 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:51.667789 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:51.680416 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:51.680392 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:19:52.100801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:19:52.100772 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f" Jul 03 14:20:04.106148 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:04.106112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:20:06.342540 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:06.342504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:20:06.442396 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:06.442360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:20:08.762178 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.762145 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x"] Jul 03 14:20:08.764477 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.764460 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.769765 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.769721 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 03 14:20:08.778508 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.778487 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x"] Jul 03 14:20:08.811452 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.811611 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnm6n\" (UniqueName: \"kubernetes.io/projected/209cd882-ee85-480d-8b6d-2eb4b3600677-kube-api-access-mnm6n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.811611 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811582 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209cd882-ee85-480d-8b6d-2eb4b3600677-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.811706 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.811767 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.811812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.811796 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.852920 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.852887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:20:08.912971 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.912933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.912990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnm6n\" (UniqueName: \"kubernetes.io/projected/209cd882-ee85-480d-8b6d-2eb4b3600677-kube-api-access-mnm6n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209cd882-ee85-480d-8b6d-2eb4b3600677-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913314 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913425 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.913496 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.913479 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.915232 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.915214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/209cd882-ee85-480d-8b6d-2eb4b3600677-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.915605 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.915587 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/209cd882-ee85-480d-8b6d-2eb4b3600677-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:08.922193 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:08.922170 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnm6n\" (UniqueName: \"kubernetes.io/projected/209cd882-ee85-480d-8b6d-2eb4b3600677-kube-api-access-mnm6n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x\" (UID: \"209cd882-ee85-480d-8b6d-2eb4b3600677\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:09.074807 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:09.074773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:09.219136 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:09.219113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x"] Jul 03 14:20:09.221574 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:20:09.221546 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209cd882_ee85_480d_8b6d_2eb4b3600677.slice/crio-ad67c3a76c76fa93c4175cfd0d01dbfcf11c4dc9b7e6f325f9edb2a028913334 WatchSource:0}: Error finding container ad67c3a76c76fa93c4175cfd0d01dbfcf11c4dc9b7e6f325f9edb2a028913334: Status 404 returned error can't find the container with id ad67c3a76c76fa93c4175cfd0d01dbfcf11c4dc9b7e6f325f9edb2a028913334 Jul 03 14:20:09.222695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:09.222678 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:20:10.178300 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:10.178261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" event={"ID":"209cd882-ee85-480d-8b6d-2eb4b3600677","Type":"ContainerStarted","Data":"8b4b7b17fdef047e4e8f02f1de0625966e45c5782ffdc3f0ff3b6365b87a7006"} Jul 03 14:20:10.178688 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:10.178323 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" event={"ID":"209cd882-ee85-480d-8b6d-2eb4b3600677","Type":"ContainerStarted","Data":"ad67c3a76c76fa93c4175cfd0d01dbfcf11c4dc9b7e6f325f9edb2a028913334"} Jul 03 14:20:10.199509 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:10.199463 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" podStartSLOduration=1.9807344119999999 podStartE2EDuration="2.199448074s" totalImagesPullingTime="218.713662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:20:10.19852801 +0000 UTC m=+703.620292491" watchObservedRunningTime="2026-07-03 14:20:10.199448074 +0000 UTC m=+703.621212555" Jul 03 14:20:14.203409 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:14.203369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:20:19.075527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:19.075434 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:19.076034 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:19.075542 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:19.091760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:19.091710 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:19.247273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:19.247245 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x" Jul 03 14:20:30.681222 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:20:30.681180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:26.168431 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:26.168396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:26.272184 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:26.272155 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:39.249239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.249205 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d446b5956-9mdvp"] Jul 03 14:21:39.251603 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.251582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.265969 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.265948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d446b5956-9mdvp"] Jul 03 14:21:39.274838 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.274814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-tls-cert\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.274935 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.274885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzcwm\" (UniqueName: \"kubernetes.io/projected/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-kube-api-access-xzcwm\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.375792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.375759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-tls-cert\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.375979 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.375850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzcwm\" (UniqueName: \"kubernetes.io/projected/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-kube-api-access-xzcwm\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.378212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.378183 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-tls-cert\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.391673 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.391647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzcwm\" (UniqueName: \"kubernetes.io/projected/22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3-kube-api-access-xzcwm\") pod \"authorino-6d446b5956-9mdvp\" (UID: \"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3\") " pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.560331 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.560243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d446b5956-9mdvp" Jul 03 14:21:39.591843 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.591796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:39.684433 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.684403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:39.711607 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:39.711578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d446b5956-9mdvp"] Jul 03 14:21:39.713788 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:21:39.713760 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22141a21_6cee_4b3e_b2fd_0b3c06fbd2e3.slice/crio-17c9eca9cfe6cd19fdaa2095b594752d7e191620e78ff4bb33eeed762f620ea5 WatchSource:0}: Error finding container 17c9eca9cfe6cd19fdaa2095b594752d7e191620e78ff4bb33eeed762f620ea5: Status 404 returned error can't find the container with id 17c9eca9cfe6cd19fdaa2095b594752d7e191620e78ff4bb33eeed762f620ea5 Jul 03 14:21:40.643886 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.643843 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d446b5956-9mdvp" event={"ID":"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3","Type":"ContainerStarted","Data":"5bc6fa8676365e1ae1ba77dfae7ea588b2bad96c04ab6fba3a3af6300f4badcb"} Jul 03 14:21:40.644416 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.643898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d446b5956-9mdvp" event={"ID":"22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3","Type":"ContainerStarted","Data":"17c9eca9cfe6cd19fdaa2095b594752d7e191620e78ff4bb33eeed762f620ea5"} Jul 03 14:21:40.678971 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.678928 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:40.691667 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.691608 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d446b5956-9mdvp" podStartSLOduration=1.382291001 podStartE2EDuration="1.691587354s" totalImagesPullingTime="309.296353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:21:40.689680882 +0000 UTC m=+794.111445364" watchObservedRunningTime="2026-07-03 14:21:40.691587354 +0000 UTC m=+794.113351837" Jul 03 14:21:40.745586 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.745553 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:21:40.745842 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.745802 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-54f496569c-h2tcc" podUID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" containerName="authorino" containerID="cri-o://aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367" gracePeriod=30 Jul 03 14:21:40.820917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:40.820887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:21:41.011062 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.011039 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:21:41.091999 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.091953 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp6j9\" (UniqueName: \"kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9\") pod \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " Jul 03 14:21:41.092179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.092139 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert\") pod \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\" (UID: \"c19c3b6f-1618-4e3b-b387-4f13ebcd0140\") " Jul 03 14:21:41.094179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.094139 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9" (OuterVolumeSpecName: "kube-api-access-fp6j9") pod "c19c3b6f-1618-4e3b-b387-4f13ebcd0140" (UID: "c19c3b6f-1618-4e3b-b387-4f13ebcd0140"). InnerVolumeSpecName "kube-api-access-fp6j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:21:41.102442 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.102409 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c19c3b6f-1618-4e3b-b387-4f13ebcd0140" (UID: "c19c3b6f-1618-4e3b-b387-4f13ebcd0140"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:21:41.193795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.193668 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp6j9\" (UniqueName: \"kubernetes.io/projected/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-kube-api-access-fp6j9\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:21:41.193795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.193697 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c19c3b6f-1618-4e3b-b387-4f13ebcd0140-tls-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:21:41.653560 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.652395 2580 generic.go:356] "Generic (PLEG): container finished" podID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" containerID="aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367" exitCode=0 Jul 03 14:21:41.653560 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.653126 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f496569c-h2tcc" Jul 03 14:21:41.654162 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.654136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f496569c-h2tcc" event={"ID":"c19c3b6f-1618-4e3b-b387-4f13ebcd0140","Type":"ContainerDied","Data":"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367"} Jul 03 14:21:41.654303 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.654286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f496569c-h2tcc" event={"ID":"c19c3b6f-1618-4e3b-b387-4f13ebcd0140","Type":"ContainerDied","Data":"80e825272a89e715208b820aa0aa6dfa3401d2892227c00fdd8d3afa66bfc0ae"} Jul 03 14:21:41.654402 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.654389 2580 scope.go:117] "RemoveContainer" containerID="aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367" Jul 03 14:21:41.672468 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.672439 2580 scope.go:117] "RemoveContainer" containerID="aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367" Jul 03 14:21:41.674351 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:21:41.674327 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367\": container with ID starting with aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367 not found: ID does not exist" containerID="aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367" Jul 03 14:21:41.674594 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.674571 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367"} err="failed to get container status \"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367\": rpc error: code = NotFound desc = could not find container \"aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367\": container with ID starting with aacccfed271f04c9a660d2b5c6434c63e39ab15c78114ca1b15f2a0dc5c69367 not found: ID does not exist" Jul 03 14:21:41.689052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.689026 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:21:41.696482 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:41.696456 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54f496569c-h2tcc"] Jul 03 14:21:43.205503 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:21:43.205472 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" path="/var/lib/kubelet/pods/c19c3b6f-1618-4e3b-b387-4f13ebcd0140/volumes" Jul 03 14:22:25.880498 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:22:25.880464 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:22:34.611459 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:22:34.611428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:03.539389 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:03.539329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:03.628019 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:03.627980 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:23:03.628204 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:03.628182 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" podUID="6846ac31-29b7-41df-808c-458439aea7f6" containerName="manager" containerID="cri-o://dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4" gracePeriod=30 Jul 03 14:23:03.877855 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:03.877834 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:23:04.005908 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.005872 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb5mj\" (UniqueName: \"kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj\") pod \"6846ac31-29b7-41df-808c-458439aea7f6\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " Jul 03 14:23:04.006062 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.005937 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert\") pod \"6846ac31-29b7-41df-808c-458439aea7f6\" (UID: \"6846ac31-29b7-41df-808c-458439aea7f6\") " Jul 03 14:23:04.007894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.007869 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6846ac31-29b7-41df-808c-458439aea7f6" (UID: "6846ac31-29b7-41df-808c-458439aea7f6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:23:04.008115 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.008086 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj" (OuterVolumeSpecName: "kube-api-access-fb5mj") pod "6846ac31-29b7-41df-808c-458439aea7f6" (UID: "6846ac31-29b7-41df-808c-458439aea7f6"). InnerVolumeSpecName "kube-api-access-fb5mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:23:04.083512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.083475 2580 generic.go:356] "Generic (PLEG): container finished" podID="6846ac31-29b7-41df-808c-458439aea7f6" containerID="dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4" exitCode=0 Jul 03 14:23:04.083657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.083520 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" Jul 03 14:23:04.083657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.083551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" event={"ID":"6846ac31-29b7-41df-808c-458439aea7f6","Type":"ContainerDied","Data":"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4"} Jul 03 14:23:04.083657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.083600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bfc956fc6-9lczt" event={"ID":"6846ac31-29b7-41df-808c-458439aea7f6","Type":"ContainerDied","Data":"27c5e1d41f1cfb57369211bec4efc522bfc9d47df3e22649d50c14e254cc7e23"} Jul 03 14:23:04.083657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.083629 2580 scope.go:117] "RemoveContainer" containerID="dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4" Jul 03 14:23:04.093240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.093217 2580 scope.go:117] "RemoveContainer" containerID="dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4" Jul 03 14:23:04.093525 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:23:04.093507 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4\": container with ID starting with dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4 not found: ID does not exist" containerID="dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4" Jul 03 14:23:04.093591 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.093535 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4"} err="failed to get container status \"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4\": rpc error: code = NotFound desc = could not find container \"dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4\": container with ID starting with dd0b3829c1c876d97b7a3036e679a8f01c7eac98f77347d5c508fa83540194e4 not found: ID does not exist" Jul 03 14:23:04.106527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.106505 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6846ac31-29b7-41df-808c-458439aea7f6-webhook-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:23:04.106527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.106523 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb5mj\" (UniqueName: \"kubernetes.io/projected/6846ac31-29b7-41df-808c-458439aea7f6-kube-api-access-fb5mj\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:23:04.121493 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.121472 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:23:04.127164 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.127144 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bfc956fc6-9lczt"] Jul 03 14:23:04.803431 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:04.803401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:05.205327 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:05.205296 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6846ac31-29b7-41df-808c-458439aea7f6" path="/var/lib/kubelet/pods/6846ac31-29b7-41df-808c-458439aea7f6/volumes" Jul 03 14:23:41.296254 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:41.296219 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:41.519111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:41.519076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:50.692994 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:50.692966 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:51.067677 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:51.067637 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:51.418171 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:51.418091 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:58.366101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:58.366066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:23:59.576078 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:23:59.576047 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:24:36.388374 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:24:36.388335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:24:36.573899 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:24:36.573868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:25:04.210161 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:25:04.210082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:27:42.181559 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:42.181521 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:27:57.561601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:57.561514 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:27:57.562019 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:57.561783 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" podUID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" containerName="manager" containerID="cri-o://73d387ded50f3a67a571310744e4221eab5890a311ac38f4e9635b4a6b57a181" gracePeriod=10 Jul 03 14:27:59.603379 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.603345 2580 generic.go:356] "Generic (PLEG): container finished" podID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" containerID="73d387ded50f3a67a571310744e4221eab5890a311ac38f4e9635b4a6b57a181" exitCode=0 Jul 03 14:27:59.603755 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.603415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" event={"ID":"adff8f2a-68aa-4fde-82cd-81d5db73b7e3","Type":"ContainerDied","Data":"73d387ded50f3a67a571310744e4221eab5890a311ac38f4e9635b4a6b57a181"} Jul 03 14:27:59.718053 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.718032 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:27:59.800052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.799946 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume\") pod \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " Jul 03 14:27:59.800052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.799999 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k7fw\" (UniqueName: \"kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw\") pod \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\" (UID: \"adff8f2a-68aa-4fde-82cd-81d5db73b7e3\") " Jul 03 14:27:59.800387 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.800366 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "adff8f2a-68aa-4fde-82cd-81d5db73b7e3" (UID: "adff8f2a-68aa-4fde-82cd-81d5db73b7e3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:27:59.801966 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.801946 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw" (OuterVolumeSpecName: "kube-api-access-8k7fw") pod "adff8f2a-68aa-4fde-82cd-81d5db73b7e3" (UID: "adff8f2a-68aa-4fde-82cd-81d5db73b7e3"). InnerVolumeSpecName "kube-api-access-8k7fw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:27:59.901372 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.901336 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-extensions-socket-volume\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:27:59.901372 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:27:59.901366 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k7fw\" (UniqueName: \"kubernetes.io/projected/adff8f2a-68aa-4fde-82cd-81d5db73b7e3-kube-api-access-8k7fw\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:28:00.609343 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:00.609317 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" Jul 03 14:28:00.609787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:00.609361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2" event={"ID":"adff8f2a-68aa-4fde-82cd-81d5db73b7e3","Type":"ContainerDied","Data":"a9b149eb79083ad637cc786f87d3e68b23302b46aa178889c8454f543cff004e"} Jul 03 14:28:00.609787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:00.609423 2580 scope.go:117] "RemoveContainer" containerID="73d387ded50f3a67a571310744e4221eab5890a311ac38f4e9635b4a6b57a181" Jul 03 14:28:00.632396 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:00.632373 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:28:00.636635 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:00.636613 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q4zx2"] Jul 03 14:28:01.205336 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:01.205306 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" path="/var/lib/kubelet/pods/adff8f2a-68aa-4fde-82cd-81d5db73b7e3/volumes" Jul 03 14:28:58.744070 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744034 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557"] Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744369 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" containerName="authorino" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744379 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" containerName="authorino" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744397 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6846ac31-29b7-41df-808c-458439aea7f6" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744403 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6846ac31-29b7-41df-808c-458439aea7f6" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744413 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744419 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744474 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="adff8f2a-68aa-4fde-82cd-81d5db73b7e3" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744481 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6846ac31-29b7-41df-808c-458439aea7f6" containerName="manager" Jul 03 14:28:58.746370 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.744488 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19c3b6f-1618-4e3b-b387-4f13ebcd0140" containerName="authorino" Jul 03 14:28:58.747292 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.747277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.750133 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.750112 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kd9nd\"" Jul 03 14:28:58.760624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.760602 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557"] Jul 03 14:28:58.791262 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.791227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgvsf\" (UniqueName: \"kubernetes.io/projected/291c995d-ae5e-4545-93ec-c678c8ef9f92-kube-api-access-lgvsf\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.791378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.791284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291c995d-ae5e-4545-93ec-c678c8ef9f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.892690 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.892653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291c995d-ae5e-4545-93ec-c678c8ef9f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.892902 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.892756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgvsf\" (UniqueName: \"kubernetes.io/projected/291c995d-ae5e-4545-93ec-c678c8ef9f92-kube-api-access-lgvsf\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.893094 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.893069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291c995d-ae5e-4545-93ec-c678c8ef9f92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:58.903227 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:58.903196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgvsf\" (UniqueName: \"kubernetes.io/projected/291c995d-ae5e-4545-93ec-c678c8ef9f92-kube-api-access-lgvsf\") pod \"kuadrant-operator-controller-manager-86878c769f-6f557\" (UID: \"291c995d-ae5e-4545-93ec-c678c8ef9f92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:59.057502 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.057412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:59.196470 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.196432 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557"] Jul 03 14:28:59.199796 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:28:59.199766 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291c995d_ae5e_4545_93ec_c678c8ef9f92.slice/crio-1c751dc111082f100f8bb422dbf0afbc02787402156a12d7bf3aea865f91ddff WatchSource:0}: Error finding container 1c751dc111082f100f8bb422dbf0afbc02787402156a12d7bf3aea865f91ddff: Status 404 returned error can't find the container with id 1c751dc111082f100f8bb422dbf0afbc02787402156a12d7bf3aea865f91ddff Jul 03 14:28:59.902996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.902957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" event={"ID":"291c995d-ae5e-4545-93ec-c678c8ef9f92","Type":"ContainerStarted","Data":"df172c35495fbc08cbfc9d7ed2ffc14382f413fc01e2ea0b053741ad769d9530"} Jul 03 14:28:59.903459 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.903008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" event={"ID":"291c995d-ae5e-4545-93ec-c678c8ef9f92","Type":"ContainerStarted","Data":"1c751dc111082f100f8bb422dbf0afbc02787402156a12d7bf3aea865f91ddff"} Jul 03 14:28:59.903459 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.903039 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:28:59.927181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:28:59.927125 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" podStartSLOduration=1.927109222 podStartE2EDuration="1.927109222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:28:59.924718356 +0000 UTC m=+1233.346482838" watchObservedRunningTime="2026-07-03 14:28:59.927109222 +0000 UTC m=+1233.348873703" Jul 03 14:29:10.909812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:29:10.909781 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f557" Jul 03 14:30:00.166280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.166240 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-slngv"] Jul 03 14:30:00.169556 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.169537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:00.172929 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.172909 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f66fv\"" Jul 03 14:30:00.181225 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.181200 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-slngv"] Jul 03 14:30:00.314375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.314335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49pml\" (UniqueName: \"kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml\") pod \"maas-api-key-cleanup-29718150-slngv\" (UID: \"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c\") " pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:00.415433 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.415393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49pml\" (UniqueName: \"kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml\") pod \"maas-api-key-cleanup-29718150-slngv\" (UID: \"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c\") " pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:00.424503 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.424438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49pml\" (UniqueName: \"kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml\") pod \"maas-api-key-cleanup-29718150-slngv\" (UID: \"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c\") " pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:00.481191 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.481148 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:00.625399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.625370 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718150-slngv"] Jul 03 14:30:00.628130 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:30:00.628103 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfd1bfc_0933_40c6_82c8_410a3e3fc04c.slice/crio-1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0 WatchSource:0}: Error finding container 1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0: Status 404 returned error can't find the container with id 1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0 Jul 03 14:30:00.629990 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:00.629976 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:30:01.218286 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:01.218250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" event={"ID":"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c","Type":"ContainerStarted","Data":"1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0"} Jul 03 14:30:03.230224 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:03.230195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" event={"ID":"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c","Type":"ContainerStarted","Data":"a23fcfa135d62256038794f0541acc66a28da8046255549e2638e908191dfd4c"} Jul 03 14:30:03.248899 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:03.248849 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" podStartSLOduration=0.983910063 podStartE2EDuration="3.248834108s" totalImagesPullingTime="2.264924045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:30:03.24740795 +0000 UTC m=+1296.669172435" watchObservedRunningTime="2026-07-03 14:30:03.248834108 +0000 UTC m=+1296.670598589" Jul 03 14:30:04.237133 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:04.237094 2580 generic.go:356] "Generic (PLEG): container finished" podID="9cfd1bfc-0933-40c6-82c8-410a3e3fc04c" containerID="a23fcfa135d62256038794f0541acc66a28da8046255549e2638e908191dfd4c" exitCode=0 Jul 03 14:30:04.237512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:04.237155 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" event={"ID":"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c","Type":"ContainerDied","Data":"a23fcfa135d62256038794f0541acc66a28da8046255549e2638e908191dfd4c"} Jul 03 14:30:05.381305 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:05.381281 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:30:05.453549 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:05.453514 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49pml\" (UniqueName: \"kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml\") pod \"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c\" (UID: \"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c\") " Jul 03 14:30:05.455475 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:05.455445 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml" (OuterVolumeSpecName: "kube-api-access-49pml") pod "9cfd1bfc-0933-40c6-82c8-410a3e3fc04c" (UID: "9cfd1bfc-0933-40c6-82c8-410a3e3fc04c"). InnerVolumeSpecName "kube-api-access-49pml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:30:05.554627 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:05.554545 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49pml\" (UniqueName: \"kubernetes.io/projected/9cfd1bfc-0933-40c6-82c8-410a3e3fc04c-kube-api-access-49pml\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:30:06.249868 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:06.249834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" event={"ID":"9cfd1bfc-0933-40c6-82c8-410a3e3fc04c","Type":"ContainerDied","Data":"1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0"} Jul 03 14:30:06.250043 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:06.249880 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d101a3d20723833ee12ad721eb391f68db237b5c91713dd375b9feda17003f0" Jul 03 14:30:06.250043 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:30:06.249884 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718150-slngv" Jul 03 14:31:19.812015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:31:19.811923 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:31:58.769200 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:31:58.769158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:02.579390 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:02.579358 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:49.477642 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:49.477612 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:49.575357 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:49.575315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:49.956232 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:49.956198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:55.805486 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:55.805453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:55.905999 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:55.905960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:56.053094 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:56.053044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:56.135358 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:56.135328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.244571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.244527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.332278 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.332239 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.403232 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.403206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.622685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.622650 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.820110 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.820080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:57.951462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:57.951374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:34:59.985823 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:34:59.985783 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:02.442099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:02.442061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:02.620506 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:02.620477 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:02.718894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:02.718815 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:02.819390 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:02.819361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:04.639140 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:04.639102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:08.714714 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:08.714683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:08.807977 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:08.807940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:08.969090 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:08.968994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:09.095830 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:09.095794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.188674 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.188634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.308969 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.308931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.403096 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.403063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.503530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.503442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.894030 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.894000 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:10.995704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:10.995667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:12.299530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:12.299504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:15.195890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:15.195857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:15.552809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:15.552695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:21.511241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:21.511139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:21.597230 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:21.597198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:23.500894 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:23.500854 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:27.388739 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:27.388698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:27.449347 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:27.449319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:27.600534 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:27.600501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:27.700407 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:27.700329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:28.787311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:28.787272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:28.875212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:28.875183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:28.965962 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:28.965921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:29.189911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:29.189879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:29.452618 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:29.452540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:29.553208 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:29.553182 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:30.801227 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:30.801186 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:33.283205 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:33.283169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:33.465291 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:33.465257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:33.556398 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:33.556320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:33.864536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:33.864502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:35.846682 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:35.846653 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:39.223910 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:39.223876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:39.287210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:39.287172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:39.718150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:39.718120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:39.814550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:39.814516 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:39.914070 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:39.914036 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:41.264023 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:41.263979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:47.453925 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:47.453894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:47.559368 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:47.559326 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:47.659535 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:47.659498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:47.754974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:47.754892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:35:48.170350 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:35:48.170316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:14.127802 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:14.127769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:20.315112 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:20.315077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:20.376818 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:20.376789 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:20.459338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:20.459242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:20.552614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:20.552585 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:36.418079 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:36.418048 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:42.162537 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:42.162500 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:42.263447 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:42.263418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:42.363918 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:42.363889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:52.907033 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:52.906954 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:53.662514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:53.662484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:59.443944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:59.443907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:59.503077 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:59.503047 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:59.596810 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:59.596779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:59.696221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:59.696131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:36:59.795297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:36:59.795261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:00.550978 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:00.550945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:00.671568 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:00.671527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:02.320885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:02.320850 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:08.241586 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:08.241552 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:08.340915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:08.340874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:08.904020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:08.903976 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:09.807008 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:09.806975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:14.712070 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:14.712029 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:14.766792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:14.766763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:14.951490 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:14.951455 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:15.266939 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:15.266897 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:15.543089 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:15.543006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.156012 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.155977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.245800 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.245762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.305460 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.305425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.650530 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.650502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.775988 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.775958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:18.862754 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:18.862696 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:22.688280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:22.688244 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:23.339224 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:23.339188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:23.468865 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:23.468834 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:23.767562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:23.767473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:29.523196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:29.523167 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:29.585465 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:29.585437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:29.935690 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:29.935655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:30.317257 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:30.317200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:30.387916 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:30.387884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:36.625225 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:36.625190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:36.725152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:36.725122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:36.853110 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:36.853081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:37.065828 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:37.065719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:37.198748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:37.198693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:37.289097 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:37.289063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:37.396555 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:37.396529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:43.175671 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:43.175641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:43.279036 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:43.279010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:43.399875 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:43.399844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:43.543051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:43.542974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:49.411033 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:49.410998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:49.511394 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:49.511362 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:55.804003 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:55.803968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:56.109785 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:56.109753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:56.206909 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:56.206881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:59.190703 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:59.190670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:59.275748 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:59.275704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:59.337639 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:59.337610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:59.681219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:59.681178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:37:59.978705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:37:59.978633 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:00.077034 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:00.077003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:03.711162 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:03.711127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:12.543291 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.543257 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:38:12.543657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.543613 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cfd1bfc-0933-40c6-82c8-410a3e3fc04c" containerName="cleanup" Jul 03 14:38:12.543657 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.543624 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cfd1bfc-0933-40c6-82c8-410a3e3fc04c" containerName="cleanup" Jul 03 14:38:12.543759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.543690 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cfd1bfc-0933-40c6-82c8-410a3e3fc04c" containerName="cleanup" Jul 03 14:38:12.545596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.545578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.548038 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.548006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-072d4360\"/\"openshift-service-ca.crt\"" Jul 03 14:38:12.548975 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.548955 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-072d4360\"/\"kube-root-ca.crt\"" Jul 03 14:38:12.549083 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.548955 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-072d4360\"/\"default-dockercfg-6sb9x\"" Jul 03 14:38:12.549083 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.548957 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-072d4360\"/\"e2e-lifecycle-model-072d4360-kserve-self-signed-certs\"" Jul 03 14:38:12.556853 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.556834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:38:12.565157 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.565138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:12.626078 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.626078 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.626329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.626329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626149 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.626329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k8hz\" (UniqueName: \"kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.626329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.626290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727506 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727506 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8hz\" (UniqueName: \"kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.727778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.728016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.727968 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.728071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.728004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.728111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.728071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.729831 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.729810 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.730180 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.730159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.735899 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.735882 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8hz\" (UniqueName: \"kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz\") pod \"e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.855156 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.855126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:12.988274 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:12.988241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:38:12.990960 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:38:12.990931 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0d1cb87_48fd_4ec7_807e_0a4f1cf076f1.slice/crio-11ee639bb03af83f1af24a927218219d41f1e262e96826a7f824d78fddd191c4 WatchSource:0}: Error finding container 11ee639bb03af83f1af24a927218219d41f1e262e96826a7f824d78fddd191c4: Status 404 returned error can't find the container with id 11ee639bb03af83f1af24a927218219d41f1e262e96826a7f824d78fddd191c4 Jul 03 14:38:13.772685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:13.772647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" event={"ID":"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1","Type":"ContainerStarted","Data":"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97"} Jul 03 14:38:13.772685 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:13.772689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" event={"ID":"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1","Type":"ContainerStarted","Data":"11ee639bb03af83f1af24a927218219d41f1e262e96826a7f824d78fddd191c4"} Jul 03 14:38:13.790507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:13.790461 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" podStartSLOduration=1.790444474 podStartE2EDuration="1.790444474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:38:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:38:13.789466076 +0000 UTC m=+1787.211230559" watchObservedRunningTime="2026-07-03 14:38:13.790444474 +0000 UTC m=+1787.212208957" Jul 03 14:38:22.855503 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:22.855413 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:22.855959 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:22.855689 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:22.867847 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:22.867825 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:23.837150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:23.837118 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:38:26.869848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:26.869811 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:26.970274 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:26.970242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:35.983586 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:35.983539 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:36.624125 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:36.624092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:36.743399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:36.743368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:36.987806 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:36.987711 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:42.103044 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:42.103009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:38:47.089282 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:47.089250 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:38:47.186309 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:38:47.186272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:02.109282 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.109239 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" podUID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" containerName="main" containerID="cri-o://3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97" gracePeriod=45 Jul 03 14:39:02.353987 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.353961 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:39:02.459061 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.458980 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k8hz\" (UniqueName: \"kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459061 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459017 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459161 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459213 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459237 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459285 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459251 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:39:02.459481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459300 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs\") pod \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\" (UID: \"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1\") " Jul 03 14:39:02.459481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459394 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:39:02.459481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459462 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home" (OuterVolumeSpecName: "home") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:39:02.459607 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459578 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tmp-dir\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:02.459607 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459592 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-home\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:02.459607 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.459600 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-model-cache\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:02.461144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.461119 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz" (OuterVolumeSpecName: "kube-api-access-9k8hz") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "kube-api-access-9k8hz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:39:02.461252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.461234 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 14:39:02.461326 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.461308 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm" (OuterVolumeSpecName: "dshm") pod "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" (UID: "f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:39:02.565888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.565847 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-tls-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:02.566191 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.566173 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k8hz\" (UniqueName: \"kubernetes.io/projected/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-kube-api-access-9k8hz\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:02.566340 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:02.566326 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1-dshm\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:39:03.030201 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.030158 2580 generic.go:356] "Generic (PLEG): container finished" podID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" containerID="3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97" exitCode=0 Jul 03 14:39:03.032385 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.030221 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" Jul 03 14:39:03.032385 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.030233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" event={"ID":"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1","Type":"ContainerDied","Data":"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97"} Jul 03 14:39:03.032385 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.030278 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn" event={"ID":"f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1","Type":"ContainerDied","Data":"11ee639bb03af83f1af24a927218219d41f1e262e96826a7f824d78fddd191c4"} Jul 03 14:39:03.032385 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.030295 2580 scope.go:117] "RemoveContainer" containerID="3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97" Jul 03 14:39:03.039873 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.039851 2580 scope.go:117] "RemoveContainer" containerID="3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97" Jul 03 14:39:03.040147 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:39:03.040124 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97\": container with ID starting with 3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97 not found: ID does not exist" containerID="3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97" Jul 03 14:39:03.040219 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.040159 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97"} err="failed to get container status \"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97\": rpc error: code = NotFound desc = could not find container \"3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97\": container with ID starting with 3154adfe16d36b1d28c4784fabdbeaf3c3d6f4120e3b6fcfead97289307adf97 not found: ID does not exist" Jul 03 14:39:03.051552 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.051530 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:39:03.055363 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.055337 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-072d4360/e2e-lifecycle-model-072d4360-kserve-7f96cbff87-wsmfn"] Jul 03 14:39:03.205906 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:03.205873 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" path="/var/lib/kubelet/pods/f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1/volumes" Jul 03 14:39:23.962123 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:23.962068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:24.446443 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:24.446409 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:30.250052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:30.250020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:30.321798 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:30.321773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:30.530272 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:30.530177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:30.771116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:30.771081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:31.840636 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:31.840605 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:31.941183 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:31.941149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:32.038140 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:32.038107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:32.785632 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:32.785602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:32.884344 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:32.884316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:32.983889 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:32.983856 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:34.320322 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:34.320289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:40.211051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:40.211021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:40.272899 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:40.272869 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:40.867328 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:40.867297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:45.279608 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:45.279557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:45.382261 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:45.382222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:46.762550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:46.762518 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:46.867912 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:46.867864 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:46.963111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:46.963082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:57.449329 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:57.449227 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:39:57.781300 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:39:57.781225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:18.782064 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:18.782029 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:24.699019 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:24.698982 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:24.753242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:24.753202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:25.038457 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:25.038376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:25.365558 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:25.365529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:25.612107 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:25.612055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:27.969290 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:27.969259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:28.063512 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:28.063460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:28.221758 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:28.221675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:28.484451 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:28.484354 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:28.816954 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:28.816916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:28.923380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:28.923347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:32.450938 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:32.450902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:32.787325 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:32.787223 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:33.133519 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.133484 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:40:33.133886 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.133874 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" containerName="main" Jul 03 14:40:33.133935 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.133890 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" containerName="main" Jul 03 14:40:33.133974 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.133961 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d1cb87-48fd-4ec7-807e-0a4f1cf076f1" containerName="main" Jul 03 14:40:33.137049 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.137026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.140681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.140660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7195a2-openshift-default-dockercfg-9xh6n\"" Jul 03 14:40:33.142024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.142008 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 14:40:33.163439 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.163413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:40:33.190069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.190041 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:33.236829 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.236829 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9xh\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.236991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.237017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.237221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.237044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337628 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337592 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337628 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9xh\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.337917 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.337878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.338271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.338215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.338271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.338233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.338378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.338300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.338433 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.338379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.338668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.338644 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.340120 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.340100 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.340372 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.340356 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.345963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.345938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.346109 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.346088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9xh\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh\") pod \"e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.447604 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.447522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:33.517139 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.517010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:33.589356 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.589332 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:40:33.592211 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:40:33.592177 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0374e98_9dc0_45e5_87c0_162bc2c8729f.slice/crio-f2cb1eaa25de7bbe944980756dcb6f3b23502987f2f8a8bd5394b7b0befbab13 WatchSource:0}: Error finding container f2cb1eaa25de7bbe944980756dcb6f3b23502987f2f8a8bd5394b7b0befbab13: Status 404 returned error can't find the container with id f2cb1eaa25de7bbe944980756dcb6f3b23502987f2f8a8bd5394b7b0befbab13 Jul 03 14:40:33.594346 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:33.594326 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:40:34.497313 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:34.497270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerStarted","Data":"f2cb1eaa25de7bbe944980756dcb6f3b23502987f2f8a8bd5394b7b0befbab13"} Jul 03 14:40:36.160528 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:36.160490 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:36.160809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:36.160564 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:36.160809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:36.160594 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:36.510804 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:36.510693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerStarted","Data":"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de"} Jul 03 14:40:36.534562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:36.534507 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podStartSLOduration=0.968807892 podStartE2EDuration="3.534492286s" totalImagesPullingTime="2.565684394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:40:36.532772949 +0000 UTC m=+1929.954537433" watchObservedRunningTime="2026-07-03 14:40:36.534492286 +0000 UTC m=+1929.956256766" Jul 03 14:40:37.448037 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:37.448002 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:37.453048 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:37.453026 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:37.517087 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:37.517054 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:37.518024 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:37.518003 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:38.560418 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:38.560297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:38.650199 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:38.650164 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:38.803095 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:38.803063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:39.087884 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:39.087846 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:39.249179 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:39.249144 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:39.349210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:39.349128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:43.093173 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:43.093137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:44.276210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:44.276176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:50.405881 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.405795 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:50.506501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.506466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:40:50.593802 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.593767 2580 generic.go:356] "Generic (PLEG): container finished" podID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerID="2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de" exitCode=0 Jul 03 14:40:50.593978 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.593843 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerDied","Data":"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de"} Jul 03 14:40:50.594239 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.594225 2580 scope.go:117] "RemoveContainer" containerID="2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de" Jul 03 14:40:50.596324 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.596292 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:50.596418 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.596364 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:50.596418 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:50.596396 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:40:51.605241 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:51.605191 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerStarted","Data":"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9"} Jul 03 14:40:52.448130 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:52.448097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:52.449233 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:52.449212 2580 patch_prober.go:28] interesting pod/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 03 14:40:52.449313 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:52.449258 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 03 14:40:52.517851 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:52.517812 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:53.448870 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:53.448842 2580 patch_prober.go:28] interesting pod/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 03 14:40:53.449319 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:53.449295 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 03 14:40:54.448707 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:54.448666 2580 patch_prober.go:28] interesting pod/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 03 14:40:54.448944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:54.448760 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 03 14:40:55.447890 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:55.447852 2580 patch_prober.go:28] interesting pod/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 03 14:40:55.448071 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:55.447904 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 03 14:40:56.453138 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:56.453104 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:56.453971 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:56.453947 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:40:59.984254 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:40:59.984225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:41:00.084600 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:41:00.084566 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:41:03.979105 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:41:03.979077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:41:11.469850 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:41:11.469816 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:41:11.578906 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:41:11.578868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:41:43.781428 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:41:43.781379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:28.262250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:28.262214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:29.243108 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:29.243075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:29.433332 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:29.433297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:29.809499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:29.809468 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:29.910101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:29.910067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:30.010817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:30.010785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:30.118712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:30.118685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:34.889166 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:34.889130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:40.082317 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:40.082280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:42:40.183061 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:42:40.183023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:11.844355 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:11.844282 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:12.111915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:12.111826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:12.340144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:12.340111 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:12.508955 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:12.508881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:12.902579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:12.902546 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:13.007843 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:13.007810 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:13.199598 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:13.199516 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:19.072885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.072850 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:19.134382 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.134347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:19.304686 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.304652 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:43:19.304989 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.304908 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" containerID="cri-o://5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9" gracePeriod=30 Jul 03 14:43:19.319653 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.319618 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:19.535287 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.535209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:19.870403 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:19.870357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:20.183189 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:20.183102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:20.649055 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:20.649019 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:22.782152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:22.782117 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:22.863670 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:22.863638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:22.927437 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:22.927402 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:23.076867 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:23.076837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:23.370861 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:23.370780 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:23.474877 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:23.474850 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:24.551250 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.551216 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:43:24.623841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623805 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm9xh\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.623841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623842 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623878 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623900 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623930 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623951 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.623987 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624010 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624041 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo\") pod \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\" (UID: \"b0374e98-9dc0-45e5-87c0-162bc2c8729f\") " Jul 03 14:43:24.624462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624351 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:43:24.624462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624340 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:43:24.624462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624408 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:43:24.624681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624560 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-credential-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.624681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624579 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.624681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.624593 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-workload-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.625069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.625038 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:43:24.625183 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.625160 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:43:24.626305 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.626285 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh" (OuterVolumeSpecName: "kube-api-access-mm9xh") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "kube-api-access-mm9xh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:43:24.626399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.626328 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 14:43:24.626564 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.626546 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:43:24.626564 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.626554 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0374e98-9dc0-45e5-87c0-162bc2c8729f" (UID: "b0374e98-9dc0-45e5-87c0-162bc2c8729f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:43:24.725151 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725118 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm9xh\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-kube-api-access-mm9xh\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.725151 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725147 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.725151 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725157 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-envoy\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.725406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725165 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-data\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.725406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725175 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istiod-ca-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:24.725406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:24.725183 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0374e98-9dc0-45e5-87c0-162bc2c8729f-istio-podinfo\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:43:25.392065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.392021 2580 generic.go:356] "Generic (PLEG): container finished" podID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerID="5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9" exitCode=0 Jul 03 14:43:25.392242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.392078 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" Jul 03 14:43:25.392242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.392106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerDied","Data":"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9"} Jul 03 14:43:25.392242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.392161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl" event={"ID":"b0374e98-9dc0-45e5-87c0-162bc2c8729f","Type":"ContainerDied","Data":"f2cb1eaa25de7bbe944980756dcb6f3b23502987f2f8a8bd5394b7b0befbab13"} Jul 03 14:43:25.392242 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.392184 2580 scope.go:117] "RemoveContainer" containerID="5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9" Jul 03 14:43:25.400876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.400860 2580 scope.go:117] "RemoveContainer" containerID="2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de" Jul 03 14:43:25.408982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.408964 2580 scope.go:117] "RemoveContainer" containerID="5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9" Jul 03 14:43:25.409253 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:43:25.409234 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9\": container with ID starting with 5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9 not found: ID does not exist" containerID="5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9" Jul 03 14:43:25.409342 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.409264 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9"} err="failed to get container status \"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9\": rpc error: code = NotFound desc = could not find container \"5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9\": container with ID starting with 5d443ae597b4b2c0abab32ba8142b47f2f1b58c67cd51a37c3e928f3ef1d53a9 not found: ID does not exist" Jul 03 14:43:25.409342 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.409290 2580 scope.go:117] "RemoveContainer" containerID="2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de" Jul 03 14:43:25.409533 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:43:25.409516 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de\": container with ID starting with 2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de not found: ID does not exist" containerID="2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de" Jul 03 14:43:25.409579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.409538 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de"} err="failed to get container status \"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de\": rpc error: code = NotFound desc = could not find container \"2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de\": container with ID starting with 2f82ae650b80c4f83e07a43aa0bdaed7bb3ca4ceabea9e296364510ea4d882de not found: ID does not exist" Jul 03 14:43:25.410937 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.410916 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:43:25.414371 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:25.414351 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7195a2-openshift-default-66976999bf-cp9nl"] Jul 03 14:43:27.210210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.210173 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" path="/var/lib/kubelet/pods/b0374e98-9dc0-45e5-87c0-162bc2c8729f/volumes" Jul 03 14:43:27.261859 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.261830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:27.573561 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.573531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:27.677963 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.677932 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:27.774466 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.774424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:27.999373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999299 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:43:27.999676 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999663 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:27.999720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999677 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:27.999720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999686 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:27.999720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999692 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:27.999837 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999765 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:27.999837 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:27.999776 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0374e98-9dc0-45e5-87c0-162bc2c8729f" containerName="istio-proxy" Jul 03 14:43:28.004259 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.004238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.006798 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.006774 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 14:43:28.006916 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.006873 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-86730a-openshift-default-dockercfg-r97gx\"" Jul 03 14:43:28.015111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.015087 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:43:28.032492 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.032453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:28.154720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.154904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.154904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.154904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjkps\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.154904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.155051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154941 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.155051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.154973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.155051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.155001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.155051 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.155041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.219683 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.219652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:28.256470 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256470 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256470 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjkps\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256659 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256893 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256893 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256893 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.256893 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.256866 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.257167 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.257145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.257167 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.257159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.257263 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.257239 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.257303 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.257253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.258895 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.258876 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.259021 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.259005 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.265152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.265127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjkps\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.265259 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.265180 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token\") pod \"e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.314953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.314918 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:28.460486 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.460459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:43:28.462338 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:43:28.462304 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf3da56_af76_4305_bdf4_84cfa80447d1.slice/crio-737554f58c41f1350ebc755791a20ed6ad2e38f452f2e0914100623c1fcaf640 WatchSource:0}: Error finding container 737554f58c41f1350ebc755791a20ed6ad2e38f452f2e0914100623c1fcaf640: Status 404 returned error can't find the container with id 737554f58c41f1350ebc755791a20ed6ad2e38f452f2e0914100623c1fcaf640 Jul 03 14:43:28.464558 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.464523 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:28.464662 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.464627 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:28.464742 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:28.464676 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:29.418505 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:29.418464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerStarted","Data":"8f2efc39b97f6d8685279adb372e624b425cb6794044c9210b9eae9a48fd9240"} Jul 03 14:43:29.418996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:29.418522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerStarted","Data":"737554f58c41f1350ebc755791a20ed6ad2e38f452f2e0914100623c1fcaf640"} Jul 03 14:43:29.438675 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:29.438626 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podStartSLOduration=2.438611608 podStartE2EDuration="2.438611608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:43:29.437349379 +0000 UTC m=+2102.859113873" watchObservedRunningTime="2026-07-03 14:43:29.438611608 +0000 UTC m=+2102.860376088" Jul 03 14:43:30.315982 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:30.315932 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:30.321157 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:30.321127 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:30.426762 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:30.426711 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:30.427835 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:30.427813 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:31.464625 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:31.464591 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:31.540873 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:31.540836 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:31.606457 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:31.606423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:31.766451 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:31.766372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:32.118226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:32.118193 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:32.220393 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:32.220361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:35.950668 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:35.950632 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:43.933886 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:43.933847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:44.276914 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:44.276881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:44.377246 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:44.377209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:44.526491 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:44.526459 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:46.467876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.467836 2580 patch_prober.go:28] interesting pod/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 14:43:46.469143 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.467907 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 14:43:46.533387 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.533358 2580 generic.go:356] "Generic (PLEG): container finished" podID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerID="8f2efc39b97f6d8685279adb372e624b425cb6794044c9210b9eae9a48fd9240" exitCode=0 Jul 03 14:43:46.533490 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.533398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerDied","Data":"8f2efc39b97f6d8685279adb372e624b425cb6794044c9210b9eae9a48fd9240"} Jul 03 14:43:46.533900 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.533883 2580 scope.go:117] "RemoveContainer" containerID="8f2efc39b97f6d8685279adb372e624b425cb6794044c9210b9eae9a48fd9240" Jul 03 14:43:46.534931 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.534895 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:46.535038 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.534974 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:46.535038 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:46.535007 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:47.540411 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:47.540372 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerStarted","Data":"155b1d8be0adde5214735fb763d9c9ede80975941e9169de9250d4c1cbac3434"} Jul 03 14:43:48.316057 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:48.316010 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:48.316252 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:48.316226 2580 patch_prober.go:28] interesting pod/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 03 14:43:48.316312 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:48.316292 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 03 14:43:49.316323 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:49.316284 2580 patch_prober.go:28] interesting pod/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 03 14:43:49.316720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:49.316346 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 03 14:43:50.315564 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.315527 2580 patch_prober.go:28] interesting pod/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 03 14:43:50.315773 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.315586 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 03 14:43:50.318969 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.318946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:50.419835 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.419794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:50.564301 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.564250 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:50.781052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.781020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:50.946818 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:50.946787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:51.315591 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.315558 2580 patch_prober.go:28] interesting pod/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 03 14:43:51.315832 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.315648 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 03 14:43:51.340501 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.340445 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:51.453628 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.453589 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:51.544074 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.544028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:51.612406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:51.612373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:52.319223 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:52.319188 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:52.319585 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:52.319562 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:52.320223 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:52.320205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:43:57.221700 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.221666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:57.285579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.285551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:57.455558 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.455525 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:43:57.455827 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.455775 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" containerID="cri-o://155b1d8be0adde5214735fb763d9c9ede80975941e9169de9250d4c1cbac3434" gracePeriod=30 Jul 03 14:43:57.477054 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.476960 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:57.730127 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:57.730037 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:58.016970 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.016867 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:58.291846 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.291768 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:43:58.294486 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.294466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.299056 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.299036 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-5f1e04-openshift-default-dockercfg-qqh7x\"" Jul 03 14:43:58.305789 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.305764 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:43:58.313128 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.313107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:58.319438 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvvqb\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319568 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319568 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319695 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319857 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319857 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.319857 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.319765 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.420841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.420795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.420841 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.420841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.420874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.420902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.420938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421101 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvvqb\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421293 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421368 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421651 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421410 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.421711 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.421694 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.423499 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.423476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.423601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.423552 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.429050 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.429024 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvvqb\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.429492 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.429470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token\") pod \"e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.604496 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.604463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:43:58.747033 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:43:58.747003 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5017ba4_502c_4fac_8247_dbf62aa10c41.slice/crio-e0d377890ba726a28660dc9af34ef323edd72cdd389d74fdadeb1816fc322074 WatchSource:0}: Error finding container e0d377890ba726a28660dc9af34ef323edd72cdd389d74fdadeb1816fc322074: Status 404 returned error can't find the container with id e0d377890ba726a28660dc9af34ef323edd72cdd389d74fdadeb1816fc322074 Jul 03 14:43:58.747673 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.747572 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:43:58.749257 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.749221 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:58.749330 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.749304 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:58.749368 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.749346 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:43:58.841518 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:58.841489 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:43:59.605590 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:59.605550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerStarted","Data":"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500"} Jul 03 14:43:59.606015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:59.605619 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerStarted","Data":"e0d377890ba726a28660dc9af34ef323edd72cdd389d74fdadeb1816fc322074"} Jul 03 14:43:59.627462 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:43:59.627411 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podStartSLOduration=1.627396365 podStartE2EDuration="1.627396365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:43:59.625985594 +0000 UTC m=+2133.047750112" watchObservedRunningTime="2026-07-03 14:43:59.627396365 +0000 UTC m=+2133.049160846" Jul 03 14:44:00.604899 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:00.604858 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:00.610369 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:00.610340 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:00.611218 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:00.611198 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:00.612010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:00.611988 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:01.629160 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:01.629127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:01.707226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:01.707191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:01.775088 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:01.775053 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:02.355313 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.355280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:02.464827 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.464799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:02.536334 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.536307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:02.624720 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.624626 2580 generic.go:356] "Generic (PLEG): container finished" podID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerID="155b1d8be0adde5214735fb763d9c9ede80975941e9169de9250d4c1cbac3434" exitCode=0 Jul 03 14:44:02.624891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.624716 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerDied","Data":"155b1d8be0adde5214735fb763d9c9ede80975941e9169de9250d4c1cbac3434"} Jul 03 14:44:02.624891 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.624801 2580 scope.go:117] "RemoveContainer" containerID="8f2efc39b97f6d8685279adb372e624b425cb6794044c9210b9eae9a48fd9240" Jul 03 14:44:02.742031 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.742006 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:44:02.761284 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761251 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761311 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761350 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761384 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjkps\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761440 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761423 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761446 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761493 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:44:02.761656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761506 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761585 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761612 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data\") pod \"3bf3da56-af76-4305-bdf4-84cfa80447d1\" (UID: \"3bf3da56-af76-4305-bdf4-84cfa80447d1\") " Jul 03 14:44:02.761944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761678 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:44:02.761944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761722 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:44:02.761944 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761767 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:44:02.762221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761961 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.762221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761981 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-workload-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.762221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.761995 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-credential-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.762221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.762010 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bf3da56-af76-4305-bdf4-84cfa80447d1-istiod-ca-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.762709 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.762670 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:44:02.763996 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.763971 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps" (OuterVolumeSpecName: "kube-api-access-kjkps") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "kube-api-access-kjkps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:44:02.764181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.764162 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:44:02.764348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.764327 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 14:44:02.764492 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.764469 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3bf3da56-af76-4305-bdf4-84cfa80447d1" (UID: "3bf3da56-af76-4305-bdf4-84cfa80447d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:44:02.863084 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.863048 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-podinfo\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.863084 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.863077 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.863084 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.863086 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-data\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.863084 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.863095 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bf3da56-af76-4305-bdf4-84cfa80447d1-istio-envoy\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:02.863342 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:02.863105 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjkps\" (UniqueName: \"kubernetes.io/projected/3bf3da56-af76-4305-bdf4-84cfa80447d1-kube-api-access-kjkps\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:44:03.631236 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:03.631203 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" Jul 03 14:44:03.631425 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:03.631231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr" event={"ID":"3bf3da56-af76-4305-bdf4-84cfa80447d1","Type":"ContainerDied","Data":"737554f58c41f1350ebc755791a20ed6ad2e38f452f2e0914100623c1fcaf640"} Jul 03 14:44:03.631425 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:03.631296 2580 scope.go:117] "RemoveContainer" containerID="155b1d8be0adde5214735fb763d9c9ede80975941e9169de9250d4c1cbac3434" Jul 03 14:44:03.648771 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:03.648743 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:44:03.653015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:03.652982 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-86730a-openshift-default-5ccf7dc895-bmkjr"] Jul 03 14:44:05.205543 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:05.205496 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" path="/var/lib/kubelet/pods/3bf3da56-af76-4305-bdf4-84cfa80447d1/volumes" Jul 03 14:44:06.097406 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.097372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:06.409090 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.408972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:06.681794 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.681701 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:44:06.682288 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.682267 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:44:06.682335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.682294 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:44:06.682335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.682313 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:44:06.682335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.682323 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:44:06.682469 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.682412 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:44:06.687222 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.687199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.689642 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.689623 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-28686a-openshift-default-dockercfg-kx7xh\"" Jul 03 14:44:06.697144 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.697120 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:44:06.717251 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.717221 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:06.796562 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksdbq\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.796764 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796715 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.797019 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.796794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.887244 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.887211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:06.897366 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897532 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksdbq\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897882 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897882 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897997 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.897997 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.897965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.898095 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.898005 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.898095 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.898076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.898230 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.898210 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.899907 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.899884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.900150 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.900127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.905778 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.905752 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.905973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.905954 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksdbq\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq\") pod \"e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:06.998436 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:06.998340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:07.137750 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.137709 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:44:07.139499 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:44:07.139475 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d298e8c_cf7a_43e8_8f11_0c40c032afac.slice/crio-08a3cda3f52ec18739c7a0a88ab6b3ef53ae456c2d7ae85efbc80159df9bc63a WatchSource:0}: Error finding container 08a3cda3f52ec18739c7a0a88ab6b3ef53ae456c2d7ae85efbc80159df9bc63a: Status 404 returned error can't find the container with id 08a3cda3f52ec18739c7a0a88ab6b3ef53ae456c2d7ae85efbc80159df9bc63a Jul 03 14:44:07.141478 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.141444 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:07.141575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.141510 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:07.141575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.141548 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:07.654444 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.654405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerStarted","Data":"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955"} Jul 03 14:44:07.654886 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.654472 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerStarted","Data":"08a3cda3f52ec18739c7a0a88ab6b3ef53ae456c2d7ae85efbc80159df9bc63a"} Jul 03 14:44:07.674585 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.674524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podStartSLOduration=1.674508888 podStartE2EDuration="1.674508888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:07.673571129 +0000 UTC m=+2141.095335647" watchObservedRunningTime="2026-07-03 14:44:07.674508888 +0000 UTC m=+2141.096273369" Jul 03 14:44:07.998961 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:07.998875 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:09.004228 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.004192 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:09.653253 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.653220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:09.665365 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.665328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:09.666422 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.666397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:09.731243 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.731207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:09.795930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:09.795884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:10.075909 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:10.075875 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:10.222338 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:10.222287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:10.323017 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:10.322990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:14.156876 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:14.156829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:16.652880 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:16.652844 2580 patch_prober.go:28] interesting pod/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 14:44:16.653283 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:16.652903 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 14:44:17.712424 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.712387 2580 generic.go:356] "Generic (PLEG): container finished" podID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerID="5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500" exitCode=0 Jul 03 14:44:17.712901 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.712464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerDied","Data":"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500"} Jul 03 14:44:17.712901 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.712850 2580 scope.go:117] "RemoveContainer" containerID="5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500" Jul 03 14:44:17.716555 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.716516 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:17.716777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.716608 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:17.716777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:17.716648 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:18.728750 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:18.728197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerStarted","Data":"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288"} Jul 03 14:44:19.605259 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:19.605215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:19.605639 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:19.605545 2580 patch_prober.go:28] interesting pod/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 03 14:44:19.605848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:19.605823 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 03 14:44:20.605232 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:20.605183 2580 patch_prober.go:28] interesting pod/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 03 14:44:20.605769 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:20.605260 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 03 14:44:21.220232 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:21.220198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:21.605280 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:21.605237 2580 patch_prober.go:28] interesting pod/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 03 14:44:21.605817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:21.605310 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 03 14:44:22.606638 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:22.606597 2580 patch_prober.go:28] interesting pod/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 03 14:44:22.607238 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:22.607203 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 03 14:44:23.609125 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:23.609081 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:23.609534 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:23.609401 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:23.610217 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:23.610194 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:44:23.980532 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:23.980452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:24.082345 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:24.082307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:25.252436 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.252404 2580 patch_prober.go:28] interesting pod/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:38180->10.133.0.59:15021: read: connection reset by peer" start-of-body= Jul 03 14:44:25.252840 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.252471 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:38180->10.133.0.59:15021: read: connection reset by peer" Jul 03 14:44:25.771490 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.771459 2580 generic.go:356] "Generic (PLEG): container finished" podID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerID="8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955" exitCode=0 Jul 03 14:44:25.771659 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.771534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerDied","Data":"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955"} Jul 03 14:44:25.771951 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.771937 2580 scope.go:117] "RemoveContainer" containerID="8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955" Jul 03 14:44:25.773015 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.772974 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:25.773146 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.773055 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:25.773146 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:25.773089 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 03 14:44:26.778480 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:26.778448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerStarted","Data":"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e"} Jul 03 14:44:26.999117 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:26.999067 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:26.999322 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:26.999243 2580 patch_prober.go:28] interesting pod/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 03 14:44:26.999322 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:26.999287 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 03 14:44:27.999653 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:27.999612 2580 patch_prober.go:28] interesting pod/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 03 14:44:28.000294 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:28.000259 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 03 14:44:28.999181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:28.999134 2580 patch_prober.go:28] interesting pod/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 03 14:44:28.999398 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:28.999215 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 03 14:44:29.999019 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:29.998978 2580 patch_prober.go:28] interesting pod/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 03 14:44:29.999516 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:29.999066 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 03 14:44:30.921928 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:30.921896 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:31.002920 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:31.002892 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:31.003302 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:31.003136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:31.003780 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:31.003762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:44:31.026002 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:31.025967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:44:58.287691 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:44:58.287658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:45:00.147779 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.147737 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw"] Jul 03 14:45:00.148229 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.148214 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf3da56-af76-4305-bdf4-84cfa80447d1" containerName="istio-proxy" Jul 03 14:45:00.151436 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.151417 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:00.152171 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.152149 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29718165-rfzg8"] Jul 03 14:45:00.154207 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.154191 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f66fv\"" Jul 03 14:45:00.155240 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.155226 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:00.161930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.161910 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw"] Jul 03 14:45:00.171339 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.171318 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718165-rfzg8"] Jul 03 14:45:00.174942 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.174921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsd4d\" (UniqueName: \"kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d\") pod \"maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw\" (UID: \"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:00.275665 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.275628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dll5z\" (UniqueName: \"kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z\") pod \"maas-api-key-cleanup-29718165-rfzg8\" (UID: \"df202880-cec7-4f38-9a43-4915552ecad6\") " pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:00.275665 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.275682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsd4d\" (UniqueName: \"kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d\") pod \"maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw\" (UID: \"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:00.284438 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.284398 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsd4d\" (UniqueName: \"kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d\") pod \"maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw\" (UID: \"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:00.376697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.376664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dll5z\" (UniqueName: \"kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z\") pod \"maas-api-key-cleanup-29718165-rfzg8\" (UID: \"df202880-cec7-4f38-9a43-4915552ecad6\") " pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:00.388346 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.388311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dll5z\" (UniqueName: \"kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z\") pod \"maas-api-key-cleanup-29718165-rfzg8\" (UID: \"df202880-cec7-4f38-9a43-4915552ecad6\") " pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:00.463915 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.463824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:00.471000 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.470974 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:00.605705 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.605665 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw"] Jul 03 14:45:00.609364 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:45:00.609136 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0018ed_fa0d_40d0_a4e5_4e50b71598a5.slice/crio-ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a WatchSource:0}: Error finding container ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a: Status 404 returned error can't find the container with id ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a Jul 03 14:45:00.634028 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.634006 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718165-rfzg8"] Jul 03 14:45:00.646874 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:45:00.646843 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf202880_cec7_4f38_9a43_4915552ecad6.slice/crio-70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea WatchSource:0}: Error finding container 70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea: Status 404 returned error can't find the container with id 70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea Jul 03 14:45:00.962493 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.962454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" event={"ID":"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5","Type":"ContainerStarted","Data":"e67dd4c529c9b86e9a699716c02c85b1118222e116be4143a17b4ffe79eee7fe"} Jul 03 14:45:00.962683 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.962517 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" event={"ID":"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5","Type":"ContainerStarted","Data":"ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a"} Jul 03 14:45:00.963575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.963553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" event={"ID":"df202880-cec7-4f38-9a43-4915552ecad6","Type":"ContainerStarted","Data":"70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea"} Jul 03 14:45:00.980111 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:00.980069 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" podStartSLOduration=0.98005626 podStartE2EDuration="980.05626ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:45:00.97810323 +0000 UTC m=+2194.399867715" watchObservedRunningTime="2026-07-03 14:45:00.98005626 +0000 UTC m=+2194.401820743" Jul 03 14:45:01.970143 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:01.970107 2580 generic.go:356] "Generic (PLEG): container finished" podID="df202880-cec7-4f38-9a43-4915552ecad6" containerID="e449f9d1d2146f9bb396cc9d4082ae7d3aa1d82816c0bec479dc69626c36ac2b" exitCode=0 Jul 03 14:45:01.970536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:01.970184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" event={"ID":"df202880-cec7-4f38-9a43-4915552ecad6","Type":"ContainerDied","Data":"e449f9d1d2146f9bb396cc9d4082ae7d3aa1d82816c0bec479dc69626c36ac2b"} Jul 03 14:45:01.971627 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:01.971606 2580 generic.go:356] "Generic (PLEG): container finished" podID="bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" containerID="e67dd4c529c9b86e9a699716c02c85b1118222e116be4143a17b4ffe79eee7fe" exitCode=0 Jul 03 14:45:01.971773 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:01.971663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" event={"ID":"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5","Type":"ContainerDied","Data":"e67dd4c529c9b86e9a699716c02c85b1118222e116be4143a17b4ffe79eee7fe"} Jul 03 14:45:03.118485 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.118461 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:03.148601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.148577 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:03.199016 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.198987 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsd4d\" (UniqueName: \"kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d\") pod \"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5\" (UID: \"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5\") " Jul 03 14:45:03.199196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.199023 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dll5z\" (UniqueName: \"kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z\") pod \"df202880-cec7-4f38-9a43-4915552ecad6\" (UID: \"df202880-cec7-4f38-9a43-4915552ecad6\") " Jul 03 14:45:03.201116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.201090 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d" (OuterVolumeSpecName: "kube-api-access-dsd4d") pod "bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" (UID: "bc0018ed-fa0d-40d0-a4e5-4e50b71598a5"). InnerVolumeSpecName "kube-api-access-dsd4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:45:03.201208 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.201125 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z" (OuterVolumeSpecName: "kube-api-access-dll5z") pod "df202880-cec7-4f38-9a43-4915552ecad6" (UID: "df202880-cec7-4f38-9a43-4915552ecad6"). InnerVolumeSpecName "kube-api-access-dll5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:45:03.299882 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.299787 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsd4d\" (UniqueName: \"kubernetes.io/projected/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5-kube-api-access-dsd4d\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:45:03.299882 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.299819 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dll5z\" (UniqueName: \"kubernetes.io/projected/df202880-cec7-4f38-9a43-4915552ecad6-kube-api-access-dll5z\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:45:03.983971 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.983938 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" Jul 03 14:45:03.984170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.983971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw" event={"ID":"bc0018ed-fa0d-40d0-a4e5-4e50b71598a5","Type":"ContainerDied","Data":"ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a"} Jul 03 14:45:03.984170 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.984025 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab4ca515f175b1b3e0297ddcbfd06dd40703b4d59b2df28aac1253c04c72ba3a" Jul 03 14:45:03.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.985290 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" Jul 03 14:45:03.985311 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.985307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718165-rfzg8" event={"ID":"df202880-cec7-4f38-9a43-4915552ecad6","Type":"ContainerDied","Data":"70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea"} Jul 03 14:45:03.985467 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:03.985345 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f9da3f18200d6352e48d4fede7e20e2e0f36e952af165a8a7a7bc2ca1c07ea" Jul 03 14:45:07.246040 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:07.245989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:45:45.951149 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:45:45.951119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:01.774644 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:01.774607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:03.220135 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:03.220105 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:30.474681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:30.474597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:44.922510 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:44.922474 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:44.987867 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:44.987837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:47:46.767921 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:47:46.767890 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:48:01.131952 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:48:01.131921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:48:28.372995 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:48:28.372961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:48:28.473656 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:48:28.473627 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:48:37.899861 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:48:37.899817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:48:38.769293 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:48:38.769258 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:02.271617 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:02.271581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:24.681226 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:24.681188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:24.781052 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:24.781013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:31.601020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:31.600983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:31.701183 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:31.701146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:41.372768 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:41.372713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:49:57.174371 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:49:57.174339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:50:26.587882 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:26.587805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:50:27.547925 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:27.547892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:50:28.467681 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.467645 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783418424656"] Jul 03 14:50:28.468177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468063 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df202880-cec7-4f38-9a43-4915552ecad6" containerName="cleanup" Jul 03 14:50:28.468177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468077 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="df202880-cec7-4f38-9a43-4915552ecad6" containerName="cleanup" Jul 03 14:50:28.468177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468110 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" containerName="cleanup" Jul 03 14:50:28.468177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468116 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" containerName="cleanup" Jul 03 14:50:28.468177 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468175 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="df202880-cec7-4f38-9a43-4915552ecad6" containerName="cleanup" Jul 03 14:50:28.468349 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.468183 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" containerName="cleanup" Jul 03 14:50:28.471056 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.471041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:28.476950 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.476925 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424656"] Jul 03 14:50:28.581575 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.581540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sfpr\" (UniqueName: \"kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr\") pod \"test-curl-7123-139783418424656\" (UID: \"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c\") " pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:28.682858 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.682821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sfpr\" (UniqueName: \"kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr\") pod \"test-curl-7123-139783418424656\" (UID: \"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c\") " pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:28.692041 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.692007 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sfpr\" (UniqueName: \"kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr\") pod \"test-curl-7123-139783418424656\" (UID: \"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c\") " pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:28.780760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.780657 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:28.913513 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.913483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424656"] Jul 03 14:50:28.915576 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:50:28.915546 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29bdc40f_28cb_4e8f_aa3e_e36eed99ed4c.slice/crio-a4c49d71c63774ebecbbd7b5acafc7328ff1bc0c5aec38c410cab9669b9d2327 WatchSource:0}: Error finding container a4c49d71c63774ebecbbd7b5acafc7328ff1bc0c5aec38c410cab9669b9d2327: Status 404 returned error can't find the container with id a4c49d71c63774ebecbbd7b5acafc7328ff1bc0c5aec38c410cab9669b9d2327 Jul 03 14:50:28.917335 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:28.917312 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 14:50:29.738599 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:29.738571 2580 generic.go:356] "Generic (PLEG): container finished" podID="29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" containerID="b23d044ccbbe9dda3cdb0954eb6acfab8bb11fac1e8afe21f01cd9c224687b3f" exitCode=0 Jul 03 14:50:29.738973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:29.738662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424656" event={"ID":"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c","Type":"ContainerDied","Data":"b23d044ccbbe9dda3cdb0954eb6acfab8bb11fac1e8afe21f01cd9c224687b3f"} Jul 03 14:50:29.738973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:29.738713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424656" event={"ID":"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c","Type":"ContainerStarted","Data":"a4c49d71c63774ebecbbd7b5acafc7328ff1bc0c5aec38c410cab9669b9d2327"} Jul 03 14:50:30.880617 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:30.880590 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:30.903441 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:30.903416 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783418424656_29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c/test-curl-7123-139783418424656/0.log" Jul 03 14:50:30.925898 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:30.925870 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783418424656"] Jul 03 14:50:30.931957 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:30.931933 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783418424656"] Jul 03 14:50:31.004460 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.004368 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sfpr\" (UniqueName: \"kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr\") pod \"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c\" (UID: \"29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c\") " Jul 03 14:50:31.006697 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.006665 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr" (OuterVolumeSpecName: "kube-api-access-5sfpr") pod "29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" (UID: "29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c"). InnerVolumeSpecName "kube-api-access-5sfpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:50:31.105683 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.105641 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sfpr\" (UniqueName: \"kubernetes.io/projected/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c-kube-api-access-5sfpr\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:50:31.205478 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.205441 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" path="/var/lib/kubelet/pods/29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c/volumes" Jul 03 14:50:31.750514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.750484 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424656" Jul 03 14:50:31.750514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:31.750510 2580 scope.go:117] "RemoveContainer" containerID="b23d044ccbbe9dda3cdb0954eb6acfab8bb11fac1e8afe21f01cd9c224687b3f" Jul 03 14:50:37.534020 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.533982 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783418423088"] Jul 03 14:50:37.534399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.534357 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" containerName="test-curl-7123-139783418424656" Jul 03 14:50:37.534399 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.534369 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" containerName="test-curl-7123-139783418424656" Jul 03 14:50:37.534473 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.534432 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29bdc40f-28cb-4e8f-aa3e-e36eed99ed4c" containerName="test-curl-7123-139783418424656" Jul 03 14:50:37.537266 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.537250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:37.545181 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.545007 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418423088"] Jul 03 14:50:37.663299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.663264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9zsz\" (UniqueName: \"kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz\") pod \"test-curl-7123-139783418423088\" (UID: \"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1\") " pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:37.764131 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.764103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9zsz\" (UniqueName: \"kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz\") pod \"test-curl-7123-139783418423088\" (UID: \"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1\") " pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:37.772491 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.772467 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9zsz\" (UniqueName: \"kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz\") pod \"test-curl-7123-139783418423088\" (UID: \"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1\") " pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:37.848038 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.848014 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:37.985759 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:37.985712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418423088"] Jul 03 14:50:37.987984 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:50:37.987951 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b89f269_cdc2_4424_bbcf_6c85b94e0ec1.slice/crio-b61648b899e810bbbc6f3097d4813bc45e5b3d939f51d9b945d7b4c32a07be06 WatchSource:0}: Error finding container b61648b899e810bbbc6f3097d4813bc45e5b3d939f51d9b945d7b4c32a07be06: Status 404 returned error can't find the container with id b61648b899e810bbbc6f3097d4813bc45e5b3d939f51d9b945d7b4c32a07be06 Jul 03 14:50:38.796373 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:38.796337 2580 generic.go:356] "Generic (PLEG): container finished" podID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" containerID="764ecbf07b2177756a2f3a919a6b1d3534abb38517299b50f3c1e854be461d65" exitCode=0 Jul 03 14:50:38.796709 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:38.796418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418423088" event={"ID":"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1","Type":"ContainerDied","Data":"764ecbf07b2177756a2f3a919a6b1d3534abb38517299b50f3c1e854be461d65"} Jul 03 14:50:38.796709 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:38.796461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418423088" event={"ID":"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1","Type":"ContainerStarted","Data":"b61648b899e810bbbc6f3097d4813bc45e5b3d939f51d9b945d7b4c32a07be06"} Jul 03 14:50:39.934828 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:39.934804 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:39.956614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:39.956587 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783418423088_5b89f269-cdc2-4424-bbcf-6c85b94e0ec1/test-curl-7123-139783418423088/0.log" Jul 03 14:50:39.981421 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:39.981388 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783418423088"] Jul 03 14:50:39.985339 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:39.985316 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783418423088"] Jul 03 14:50:40.086344 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:40.086314 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9zsz\" (UniqueName: \"kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz\") pod \"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1\" (UID: \"5b89f269-cdc2-4424-bbcf-6c85b94e0ec1\") " Jul 03 14:50:40.088395 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:40.088371 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz" (OuterVolumeSpecName: "kube-api-access-g9zsz") pod "5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" (UID: "5b89f269-cdc2-4424-bbcf-6c85b94e0ec1"). InnerVolumeSpecName "kube-api-access-g9zsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:50:40.187256 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:40.187219 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9zsz\" (UniqueName: \"kubernetes.io/projected/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1-kube-api-access-g9zsz\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:50:40.808879 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:40.808849 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:40.808879 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:40.808868 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b61648b899e810bbbc6f3097d4813bc45e5b3d939f51d9b945d7b4c32a07be06" Jul 03 14:50:40.818225 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:50:40.818199 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783418423088\" is forbidden: User \"system:node:ip-10-0-132-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-213.ec2.internal' and this object" podUID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" pod="opendatahub/test-curl-7123-139783418423088" Jul 03 14:50:41.205448 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:50:41.205412 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" path="/var/lib/kubelet/pods/5b89f269-cdc2-4424-bbcf-6c85b94e0ec1/volumes" Jul 03 14:51:09.867719 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:09.867683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:10.271596 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:10.271513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:10.369005 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:10.368973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:10.516021 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:10.515971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:16.303653 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.303625 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:16.366299 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.366263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:16.553771 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.553670 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:51:16.553951 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.553911 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" containerID="cri-o://f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288" gracePeriod=30 Jul 03 14:51:16.585950 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.585924 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:16.788809 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.788775 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:16.972116 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:16.972082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:17.426904 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.426873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:17.527545 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.527511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:17.630086 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.630054 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:17.771612 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.771525 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw"] Jul 03 14:51:17.774395 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.774371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:17.774521 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:17.774377 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-28686a-29718165-lpcjw"] Jul 03 14:51:19.206383 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:19.206348 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc0018ed-fa0d-40d0-a4e5-4e50b71598a5" path="/var/lib/kubelet/pods/bc0018ed-fa0d-40d0-a4e5-4e50b71598a5/volumes" Jul 03 14:51:21.815973 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.815910 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:51:21.951595 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951564 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951604 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951621 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951642 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951673 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951713 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951764 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.951812 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951798 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvvqb\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.952198 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951841 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo\") pod \"e5017ba4-502c-4fac-8247-dbf62aa10c41\" (UID: \"e5017ba4-502c-4fac-8247-dbf62aa10c41\") " Jul 03 14:51:21.952198 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.951958 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:21.952198 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952008 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:21.952198 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952049 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:21.952198 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952193 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-credential-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:21.952454 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952181 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:51:21.952454 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952213 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:21.952454 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952229 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-workload-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:21.952616 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.952490 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:21.954065 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.954039 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb" (OuterVolumeSpecName: "kube-api-access-mvvqb") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "kube-api-access-mvvqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:51:21.954187 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.954106 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:21.954397 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.954375 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 14:51:21.954471 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:21.954450 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5017ba4-502c-4fac-8247-dbf62aa10c41" (UID: "e5017ba4-502c-4fac-8247-dbf62aa10c41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:51:22.027792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.027759 2580 generic.go:356] "Generic (PLEG): container finished" podID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerID="f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288" exitCode=0 Jul 03 14:51:22.027941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.027847 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerDied","Data":"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288"} Jul 03 14:51:22.027941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.027906 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" event={"ID":"e5017ba4-502c-4fac-8247-dbf62aa10c41","Type":"ContainerDied","Data":"e0d377890ba726a28660dc9af34ef323edd72cdd389d74fdadeb1816fc322074"} Jul 03 14:51:22.027941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.027927 2580 scope.go:117] "RemoveContainer" containerID="f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288" Jul 03 14:51:22.027941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.027861 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k" Jul 03 14:51:22.039581 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.039563 2580 scope.go:117] "RemoveContainer" containerID="5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500" Jul 03 14:51:22.048624 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.048607 2580 scope.go:117] "RemoveContainer" containerID="f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288" Jul 03 14:51:22.048892 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:51:22.048872 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288\": container with ID starting with f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288 not found: ID does not exist" containerID="f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288" Jul 03 14:51:22.048953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.048900 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288"} err="failed to get container status \"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288\": rpc error: code = NotFound desc = could not find container \"f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288\": container with ID starting with f77107137a65a2ebe2b9d399835e1f79f6ad430eba6abcac8d47efe2df476288 not found: ID does not exist" Jul 03 14:51:22.048953 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.048919 2580 scope.go:117] "RemoveContainer" containerID="5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500" Jul 03 14:51:22.049152 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:51:22.049135 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500\": container with ID starting with 5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500 not found: ID does not exist" containerID="5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500" Jul 03 14:51:22.049194 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.049160 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500"} err="failed to get container status \"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500\": rpc error: code = NotFound desc = could not find container \"5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500\": container with ID starting with 5f2463bd8bd696b8a0916bda2b62b0af96f275c1cc33f722b6c595d4f682f500 not found: ID does not exist" Jul 03 14:51:22.053644 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053624 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-podinfo\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053649 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5017ba4-502c-4fac-8247-dbf62aa10c41-istiod-ca-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053663 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-data\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053674 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-envoy\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053686 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-istio-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053760 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053698 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvvqb\" (UniqueName: \"kubernetes.io/projected/e5017ba4-502c-4fac-8247-dbf62aa10c41-kube-api-access-mvvqb\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:22.053936 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.053782 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:51:22.058606 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:22.058586 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-5f1e04-openshift-default-5568f96b59-drx5k"] Jul 03 14:51:23.205518 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.205487 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" path="/var/lib/kubelet/pods/e5017ba4-502c-4fac-8247-dbf62aa10c41/volumes" Jul 03 14:51:23.484348 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.484259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:23.587221 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.587189 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:23.706571 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.706543 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:51:23.706866 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.706843 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" containerID="cri-o://f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e" gracePeriod=30 Jul 03 14:51:23.727787 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.727755 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:23.946214 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:23.946181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:24.341009 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:24.340979 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d446b5956-9mdvp_22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3/authorino/0.log" Jul 03 14:51:27.530552 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:27.530519 2580 scope.go:117] "RemoveContainer" containerID="e67dd4c529c9b86e9a699716c02c85b1118222e116be4143a17b4ffe79eee7fe" Jul 03 14:51:28.855503 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:28.855472 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sv2nm_b55c6133-29ef-43d6-b6b2-cc5a2e3038a1/manager/0.log" Jul 03 14:51:28.952684 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:28.952661 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:51:28.966069 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:28.966046 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6tpqh_29015242-3f99-4879-8658-7acf80cfa262/manager/0.log" Jul 03 14:51:29.068421 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.068391 2580 generic.go:356] "Generic (PLEG): container finished" podID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerID="f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e" exitCode=0 Jul 03 14:51:29.068579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.068440 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" Jul 03 14:51:29.068579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.068462 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerDied","Data":"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e"} Jul 03 14:51:29.068579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.068503 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl" event={"ID":"9d298e8c-cf7a-43e8-8f11-0c40c032afac","Type":"ContainerDied","Data":"08a3cda3f52ec18739c7a0a88ab6b3ef53ae456c2d7ae85efbc80159df9bc63a"} Jul 03 14:51:29.068579 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.068524 2580 scope.go:117] "RemoveContainer" containerID="f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e" Jul 03 14:51:29.077302 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.077278 2580 scope.go:117] "RemoveContainer" containerID="8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955" Jul 03 14:51:29.085939 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.085917 2580 scope.go:117] "RemoveContainer" containerID="f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e" Jul 03 14:51:29.086211 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:51:29.086190 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e\": container with ID starting with f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e not found: ID does not exist" containerID="f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e" Jul 03 14:51:29.086297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.086225 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e"} err="failed to get container status \"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e\": rpc error: code = NotFound desc = could not find container \"f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e\": container with ID starting with f52422c2fbd90456b16eba4867bacfb362ef2235dcbd7b0976f54111f720169e not found: ID does not exist" Jul 03 14:51:29.086297 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.086250 2580 scope.go:117] "RemoveContainer" containerID="8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955" Jul 03 14:51:29.086569 ip-10-0-132-213 kubenswrapper[2580]: E0703 14:51:29.086530 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955\": container with ID starting with 8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955 not found: ID does not exist" containerID="8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955" Jul 03 14:51:29.086652 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.086571 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955"} err="failed to get container status \"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955\": rpc error: code = NotFound desc = could not find container \"8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955\": container with ID starting with 8669b9fc12beea42e69f217cd92c074bd7f96421f3c3098610e99d266d9d5955 not found: ID does not exist" Jul 03 14:51:29.120745 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120696 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.120911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120796 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.120911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120817 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksdbq\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.120911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120843 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.120911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120863 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.120911 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120891 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.121176 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120917 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.121176 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.120991 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.121176 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121019 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert\") pod \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\" (UID: \"9d298e8c-cf7a-43e8-8f11-0c40c032afac\") " Jul 03 14:51:29.121323 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121178 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:29.121378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121328 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.121378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121185 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:29.121378 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121327 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:29.121525 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121477 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 14:51:29.121747 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.121703 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:29.123210 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.123183 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 14:51:29.123301 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.123281 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq" (OuterVolumeSpecName: "kube-api-access-ksdbq") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "kube-api-access-ksdbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:51:29.123395 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.123378 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 14:51:29.123444 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.123405 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d298e8c-cf7a-43e8-8f11-0c40c032afac" (UID: "9d298e8c-cf7a-43e8-8f11-0c40c032afac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 14:51:29.222262 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222237 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-token\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222264 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksdbq\" (UniqueName: \"kubernetes.io/projected/9d298e8c-cf7a-43e8-8f11-0c40c032afac-kube-api-access-ksdbq\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222279 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-workload-certs\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222291 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-podinfo\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222302 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-credential-socket\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222315 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-envoy\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222327 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istiod-ca-cert\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.222388 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.222338 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d298e8c-cf7a-43e8-8f11-0c40c032afac-istio-data\") on node \"ip-10-0-132-213.ec2.internal\" DevicePath \"\"" Jul 03 14:51:29.234313 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.234288 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718150-slngv_9cfd1bfc-0933-40c6-82c8-410a3e3fc04c/cleanup/0.log" Jul 03 14:51:29.338212 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.338136 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718165-rfzg8_df202880-cec7-4f38-9a43-4915552ecad6/cleanup/0.log" Jul 03 14:51:29.384381 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.384354 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:51:29.388308 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.388287 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-28686a-openshift-default-7cfc775447-dwqpl"] Jul 03 14:51:29.578804 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.578775 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gnfqt_04285a29-1f90-4799-af8f-1b700f09ccf2/server/0.log" Jul 03 14:51:29.692689 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.692587 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pk4xg_80c17a24-6f18-4c53-b19d-6ccddc379795/manager/0.log" Jul 03 14:51:29.925195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.925163 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-84bm4_20667d3d-6d30-4112-b1f8-f01fec9540b0/manager/0.log" Jul 03 14:51:29.931112 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:29.931092 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-84bm4_20667d3d-6d30-4112-b1f8-f01fec9540b0/copy-manifests/0.log" Jul 03 14:51:30.169858 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:30.169831 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rnhb4_f9d29f20-cb42-4b35-b8e1-b21934d8d0dd/postgres/0.log" Jul 03 14:51:30.943637 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:30.943610 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/pull/0.log" Jul 03 14:51:30.949947 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:30.949917 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/extract/0.log" Jul 03 14:51:30.956200 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:30.956182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/util/0.log" Jul 03 14:51:31.070074 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.070044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/util/0.log" Jul 03 14:51:31.076862 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.076839 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/pull/0.log" Jul 03 14:51:31.083169 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.083138 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/extract/0.log" Jul 03 14:51:31.197449 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.197372 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/util/0.log" Jul 03 14:51:31.212099 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.212056 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" path="/var/lib/kubelet/pods/9d298e8c-cf7a-43e8-8f11-0c40c032afac/volumes" Jul 03 14:51:31.212684 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.212666 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/pull/0.log" Jul 03 14:51:31.219930 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.219913 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/extract/0.log" Jul 03 14:51:31.331213 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.331182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/util/0.log" Jul 03 14:51:31.337998 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.337962 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/pull/0.log" Jul 03 14:51:31.343910 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.343888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/extract/0.log" Jul 03 14:51:31.467146 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.467059 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d446b5956-9mdvp_22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3/authorino/0.log" Jul 03 14:51:31.588632 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.588588 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-66r66_dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7/manager/0.log" Jul 03 14:51:31.701343 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.701316 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9jqbk_5cda36ff-3991-4960-b497-1bab3a623bec/manager/0.log" Jul 03 14:51:31.922864 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:31.922836 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c55gk_f669de20-7738-4a34-8010-e824aa940d01/registry-server/0.log" Jul 03 14:51:32.050741 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:32.050698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6f557_291c995d-ae5e-4545-93ec-c678c8ef9f92/manager/0.log" Jul 03 14:51:32.280375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:32.280304 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-58dtl_16ed0a24-3715-4875-9b94-d6b0f2a76953/manager/0.log" Jul 03 14:51:32.865941 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:32.865911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-7zztf_c712a588-b3cb-47a2-aa9d-7b5432602677/kube-auth-proxy/0.log" Jul 03 14:51:33.213507 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:33.213410 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6rx5j_05dd2c91-973d-43bb-b48a-4ac051a6dbd2/payload-pre-processing/0.log" Jul 03 14:51:33.764609 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:33.764581 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gc99f_ed6a1d0f-61a1-4a8b-8965-db88b54bef07/main/0.log" Jul 03 14:51:34.097369 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:34.097338 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbmk74x_209cd882-ee85-480d-8b6d-2eb4b3600677/main/0.log" Jul 03 14:51:38.023018 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.022982 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s2kmc/must-gather-b9pkt"] Jul 03 14:51:38.023375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023353 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023364 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023375 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023372 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" containerName="test-curl-7123-139783418423088" Jul 03 14:51:38.023481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023379 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" containerName="test-curl-7123-139783418423088" Jul 03 14:51:38.023481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023390 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.023481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023395 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.023481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023408 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023481 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023413 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023511 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b89f269-cdc2-4424-bbcf-6c85b94e0ec1" containerName="test-curl-7123-139783418423088" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023520 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023527 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023541 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5017ba4-502c-4fac-8247-dbf62aa10c41" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023548 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023658 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.023712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.023670 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d298e8c-cf7a-43e8-8f11-0c40c032afac" containerName="istio-proxy" Jul 03 14:51:38.026793 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.026776 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.028965 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.028947 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s2kmc\"/\"default-dockercfg-fgmwv\"" Jul 03 14:51:38.029155 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.029133 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2kmc\"/\"openshift-service-ca.crt\"" Jul 03 14:51:38.029274 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.029162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2kmc\"/\"kube-root-ca.crt\"" Jul 03 14:51:38.044669 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.044600 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:38.046712 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.046684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/must-gather-b9pkt"] Jul 03 14:51:38.092848 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.092818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-must-gather-output\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.093010 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.092876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6tt\" (UniqueName: \"kubernetes.io/projected/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-kube-api-access-ht6tt\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.138112 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.138080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" limit=1024 Jul 03 14:51:38.193792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.193719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6tt\" (UniqueName: \"kubernetes.io/projected/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-kube-api-access-ht6tt\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.193962 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.193824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-must-gather-output\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.194119 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.194104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-must-gather-output\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.207792 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.207759 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6tt\" (UniqueName: \"kubernetes.io/projected/f4493cdf-9ef3-42bc-96c6-6d5807e779cb-kube-api-access-ht6tt\") pod \"must-gather-b9pkt\" (UID: \"f4493cdf-9ef3-42bc-96c6-6d5807e779cb\") " pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.344966 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.344941 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" Jul 03 14:51:38.480382 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:38.480354 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/must-gather-b9pkt"] Jul 03 14:51:38.481642 ip-10-0-132-213 kubenswrapper[2580]: W0703 14:51:38.481613 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4493cdf_9ef3_42bc_96c6_6d5807e779cb.slice/crio-4ed673f3c58defaf9d7c078eef7710533ee63abe8b93d0c805cba5e7519baf13 WatchSource:0}: Error finding container 4ed673f3c58defaf9d7c078eef7710533ee63abe8b93d0c805cba5e7519baf13: Status 404 returned error can't find the container with id 4ed673f3c58defaf9d7c078eef7710533ee63abe8b93d0c805cba5e7519baf13 Jul 03 14:51:39.125274 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:39.125240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" event={"ID":"f4493cdf-9ef3-42bc-96c6-6d5807e779cb","Type":"ContainerStarted","Data":"4ed673f3c58defaf9d7c078eef7710533ee63abe8b93d0c805cba5e7519baf13"} Jul 03 14:51:40.134234 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.134187 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" event={"ID":"f4493cdf-9ef3-42bc-96c6-6d5807e779cb","Type":"ContainerStarted","Data":"b3af830b6c26ad8766e9fbbe4f734e34f76ef6da04ad10aaabe0acb6806c3ed1"} Jul 03 14:51:40.134713 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.134248 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" event={"ID":"f4493cdf-9ef3-42bc-96c6-6d5807e779cb","Type":"ContainerStarted","Data":"e4b43a8d0420e155f9d11bc23b414d5cf1a0dacd8492a8950cc6db5019f06d86"} Jul 03 14:51:40.153340 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.153273 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s2kmc/must-gather-b9pkt" podStartSLOduration=1.3800669970000001 podStartE2EDuration="2.153252454s" totalImagesPullingTime="773.185457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:51:40.150277072 +0000 UTC m=+2593.572041566" watchObservedRunningTime="2026-07-03 14:51:40.153252454 +0000 UTC m=+2593.575016938" Jul 03 14:51:40.803805 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.803772 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xdtqp_d02fb523-8f63-44bb-8b13-c7cb33393387/global-pull-secret-syncer/0.log" Jul 03 14:51:40.899264 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.899236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dzgm9_70d8fc23-58f1-4a6b-aeab-9e485a2bf288/konnectivity-agent/0.log" Jul 03 14:51:40.997418 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:40.997390 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-213.ec2.internal_6e2f5c65747e411c473e749cc46d5684/haproxy/0.log" Jul 03 14:51:45.602030 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.601957 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/extract/0.log" Jul 03 14:51:45.624795 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.624691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/util/0.log" Jul 03 14:51:45.647659 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.647629 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pkfp4_409a5eb6-527d-4915-b6a2-b0a976726af0/pull/0.log" Jul 03 14:51:45.673952 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.673926 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/extract/0.log" Jul 03 14:51:45.694271 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.694243 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/util/0.log" Jul 03 14:51:45.714302 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.714275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0648p8_bcc27fa0-cb5b-4f48-abfd-3784460fbc5c/pull/0.log" Jul 03 14:51:45.742550 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.742524 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/extract/0.log" Jul 03 14:51:45.762897 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.762863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/util/0.log" Jul 03 14:51:45.783380 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.783345 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73thl4b_56eabe35-4d42-43f3-9597-3688a32b8f62/pull/0.log" Jul 03 14:51:45.812004 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.811965 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/extract/0.log" Jul 03 14:51:45.835070 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.835039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/util/0.log" Jul 03 14:51:45.861615 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:45.861532 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128dz7_fdcf8e70-fa87-4f7e-bd6a-35b4d84072cc/pull/0.log" Jul 03 14:51:46.271801 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.271023 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d446b5956-9mdvp_22141a21-6cee-4b3e-b2fd-0b3c06fbd2e3/authorino/0.log" Jul 03 14:51:46.303459 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.303270 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-66r66_dbda8472-972e-4ff4-b5a8-8b6c8b9b87d7/manager/0.log" Jul 03 14:51:46.334235 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.334204 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9jqbk_5cda36ff-3991-4960-b497-1bab3a623bec/manager/0.log" Jul 03 14:51:46.408282 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.408225 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c55gk_f669de20-7738-4a34-8010-e824aa940d01/registry-server/0.log" Jul 03 14:51:46.575583 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.575540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6f557_291c995d-ae5e-4545-93ec-c678c8ef9f92/manager/0.log" Jul 03 14:51:46.684781 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:46.684748 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-58dtl_16ed0a24-3715-4875-9b94-d6b0f2a76953/manager/0.log" Jul 03 14:51:48.972152 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:48.972025 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtcxj_3238b6ed-5d74-4933-a005-ebf6eeab8eca/node-exporter/0.log" Jul 03 14:51:48.992278 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:48.992246 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtcxj_3238b6ed-5d74-4933-a005-ebf6eeab8eca/kube-rbac-proxy/0.log" Jul 03 14:51:49.017777 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:49.017662 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtcxj_3238b6ed-5d74-4933-a005-ebf6eeab8eca/init-textfile/0.log" Jul 03 14:51:51.111455 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.111424 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb"] Jul 03 14:51:51.121374 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.121342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.129684 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.129649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb"] Jul 03 14:51:51.245517 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.245473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-sys\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.245704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.245559 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msf22\" (UniqueName: \"kubernetes.io/projected/21df69fe-7c92-4a74-b488-52792e0e753c-kube-api-access-msf22\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.245704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.245605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-proc\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.245704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.245663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-lib-modules\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.245704 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.245689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-podres\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347093 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msf22\" (UniqueName: \"kubernetes.io/projected/21df69fe-7c92-4a74-b488-52792e0e753c-kube-api-access-msf22\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-proc\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-lib-modules\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-podres\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347273 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-sys\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-sys\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347411 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-podres\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347453 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-proc\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.347514 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.347510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21df69fe-7c92-4a74-b488-52792e0e753c-lib-modules\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.354635 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.354603 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msf22\" (UniqueName: \"kubernetes.io/projected/21df69fe-7c92-4a74-b488-52792e0e753c-kube-api-access-msf22\") pod \"perf-node-gather-daemonset-nbwbb\" (UID: \"21df69fe-7c92-4a74-b488-52792e0e753c\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.450553 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.450472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:51.620861 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.620517 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb"] Jul 03 14:51:51.836201 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:51.836170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d86bb67dd-27tqp_ebdaad94-090d-49b5-bae4-f9f354706dcd/console/0.log" Jul 03 14:51:52.239819 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:52.239704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" event={"ID":"21df69fe-7c92-4a74-b488-52792e0e753c","Type":"ContainerStarted","Data":"123596a820c52ff5ff2a8179d56fba0422e5d6080319d962ae5ca911df92f3ef"} Jul 03 14:51:52.239819 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:52.239802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" event={"ID":"21df69fe-7c92-4a74-b488-52792e0e753c","Type":"ContainerStarted","Data":"879645348f17f7bb8983f6115153b689d9775e981c4fb108e623e71c5d5e1ecd"} Jul 03 14:51:52.240333 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:52.240154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:52.260493 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:52.260427 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" podStartSLOduration=1.2604065150000001 podStartE2EDuration="1.260406515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:51:52.259336493 +0000 UTC m=+2605.681100975" watchObservedRunningTime="2026-07-03 14:51:52.260406515 +0000 UTC m=+2605.682170997" Jul 03 14:51:52.448442 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:52.448404 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-fqsxh_641f01c7-e593-41c8-bf48-9f1e8d4c24e7/volume-data-source-validator/0.log" Jul 03 14:51:53.580260 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:53.580228 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j74ld_d89c1a98-bdb5-4a93-972b-dce4073fbf34/dns/0.log" Jul 03 14:51:53.598888 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:53.598849 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j74ld_d89c1a98-bdb5-4a93-972b-dce4073fbf34/kube-rbac-proxy/0.log" Jul 03 14:51:53.708766 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:53.708742 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4tvl_f16f0567-1d4e-49f6-ad00-f4185dd5eef4/dns-node-resolver/0.log" Jul 03 14:51:54.307195 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:54.307165 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r6zkn_08296dc8-7d3d-405b-b524-3e13780e3081/node-ca/0.log" Jul 03 14:51:55.458578 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:55.458548 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-7zztf_c712a588-b3cb-47a2-aa9d-7b5432602677/kube-auth-proxy/0.log" Jul 03 14:51:55.601336 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:55.601303 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6rx5j_05dd2c91-973d-43bb-b48a-4ac051a6dbd2/payload-pre-processing/0.log" Jul 03 14:51:56.363044 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.363015 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pkcmz_5f47ef9d-ee06-4552-877d-fcc4ebc14b8c/serve-healthcheck-canary/0.log" Jul 03 14:51:56.879601 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.879568 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ccctb_bde77271-76bc-428a-a4c4-f096114b6526/insights-operator/0.log" Jul 03 14:51:56.880196 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.880145 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ccctb_bde77271-76bc-428a-a4c4-f096114b6526/insights-operator/1.log" Jul 03 14:51:56.902817 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.902791 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4s4gl_462292be-e44f-4b52-9608-0cb0207499f9/kube-rbac-proxy/0.log" Jul 03 14:51:56.924095 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.924072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4s4gl_462292be-e44f-4b52-9608-0cb0207499f9/exporter/0.log" Jul 03 14:51:56.944412 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:56.944386 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4s4gl_462292be-e44f-4b52-9608-0cb0207499f9/extractor/0.log" Jul 03 14:51:58.258450 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:58.258422 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-nbwbb" Jul 03 14:51:59.187458 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.187420 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sv2nm_b55c6133-29ef-43d6-b6b2-cc5a2e3038a1/manager/0.log" Jul 03 14:51:59.354757 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.354700 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6tpqh_29015242-3f99-4879-8658-7acf80cfa262/manager/0.log" Jul 03 14:51:59.424255 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.424230 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718150-slngv_9cfd1bfc-0933-40c6-82c8-410a3e3fc04c/cleanup/0.log" Jul 03 14:51:59.449614 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.449533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718165-rfzg8_df202880-cec7-4f38-9a43-4915552ecad6/cleanup/0.log" Jul 03 14:51:59.617618 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.617582 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gnfqt_04285a29-1f90-4799-af8f-1b700f09ccf2/server/0.log" Jul 03 14:51:59.708627 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.708548 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pk4xg_80c17a24-6f18-4c53-b19d-6ccddc379795/manager/0.log" Jul 03 14:51:59.866623 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.866582 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-84bm4_20667d3d-6d30-4112-b1f8-f01fec9540b0/manager/0.log" Jul 03 14:51:59.891913 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.891884 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-84bm4_20667d3d-6d30-4112-b1f8-f01fec9540b0/copy-manifests/0.log" Jul 03 14:51:59.975535 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:51:59.975457 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-rnhb4_f9d29f20-cb42-4b35-b8e1-b21934d8d0dd/postgres/0.log" Jul 03 14:52:01.359885 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:01.359846 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-7zztf_c712a588-b3cb-47a2-aa9d-7b5432602677/kube-auth-proxy/0.log" Jul 03 14:52:01.451623 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:01.451592 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6rx5j_05dd2c91-973d-43bb-b48a-4ac051a6dbd2/payload-pre-processing/0.log" Jul 03 14:52:01.620168 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:01.620082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xc7d9_dbd1d639-fc4a-4bbd-a2a8-8ec4bc2d7548/openshift-lws-operator/0.log" Jul 03 14:52:03.000444 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:03.000419 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-7zztf_c712a588-b3cb-47a2-aa9d-7b5432602677/kube-auth-proxy/0.log" Jul 03 14:52:03.119142 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:03.118994 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6rx5j_05dd2c91-973d-43bb-b48a-4ac051a6dbd2/payload-pre-processing/0.log" Jul 03 14:52:06.655074 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:06.655049 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-nnbp7_dd7502f4-d10f-4e60-a667-53f1ec7c89ff/migrator/0.log" Jul 03 14:52:06.676875 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:06.676853 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-nnbp7_dd7502f4-d10f-4e60-a667-53f1ec7c89ff/graceful-termination/0.log" Jul 03 14:52:08.024474 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.024401 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/kube-multus-additional-cni-plugins/0.log" Jul 03 14:52:08.044386 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.044357 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/egress-router-binary-copy/0.log" Jul 03 14:52:08.064536 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.064513 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/cni-plugins/0.log" Jul 03 14:52:08.085008 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.084978 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/bond-cni-plugin/0.log" Jul 03 14:52:08.105527 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.105494 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/routeoverride-cni/0.log" Jul 03 14:52:08.126744 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.126708 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/whereabouts-cni-bincopy/0.log" Jul 03 14:52:08.146954 ip-10-0-132-213 kubenswrapper[2580]: I0703 14:52:08.146926 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4b79s_e08c0a3f-cb79-40b8-8587-ed2f2c02b7fa/whereabouts-cni/0.log"