Jul 10 00:17:07.845251 ip-10-0-142-23 systemd[1]: Starting Kubernetes Kubelet... Jul 10 00:17:08.296645 ip-10-0-142-23 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:17:08.296645 ip-10-0-142-23 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 00:17:08.296645 ip-10-0-142-23 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:17:08.296645 ip-10-0-142-23 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 00:17:08.296645 ip-10-0-142-23 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:17:08.299624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.299476 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 00:17:08.304602 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304587 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304604 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304609 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304613 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304616 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304619 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304624 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304628 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304631 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304634 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304637 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304640 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304643 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.304640 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304646 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304649 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304652 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304654 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304657 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304660 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304668 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304670 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304673 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304676 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304678 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304681 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304683 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304686 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304688 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304691 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304694 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304698 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304700 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304703 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.304965 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304705 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304708 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304710 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304712 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304715 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304717 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304720 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304722 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304725 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304727 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304730 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304733 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304735 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304738 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304741 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304744 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304748 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304751 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304754 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.305449 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304757 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304759 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304762 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304765 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304768 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304771 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304773 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304776 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304779 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304782 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304785 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304787 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304790 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304792 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304795 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304797 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304800 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304802 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304805 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304807 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.305945 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304809 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304812 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304815 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304817 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304820 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304824 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304828 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304831 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304833 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304836 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304839 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304841 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304844 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304846 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304849 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304852 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304854 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304857 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304860 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304862 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.306440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304865 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304867 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304870 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304872 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304874 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304877 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.304880 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305593 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305601 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305604 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305607 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305609 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305612 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305615 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305618 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305620 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305623 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305626 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305629 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305632 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.306910 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305635 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305637 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305640 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305642 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305645 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305648 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305650 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305653 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305656 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305658 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305661 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305664 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305666 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305669 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305672 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305674 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305676 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305679 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305681 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305684 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.307408 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305686 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305688 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305691 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305693 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305698 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305702 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305705 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305708 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305711 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305714 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305717 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305720 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305724 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305726 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305729 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305731 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305734 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305736 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305738 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.307892 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305741 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305744 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305746 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305749 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305751 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305754 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305756 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305759 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305761 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305764 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305766 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305769 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305773 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305776 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305779 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305782 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305785 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305788 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305791 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.308363 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305794 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305797 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305799 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305802 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305806 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305810 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305813 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305815 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305819 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305821 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305824 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305826 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305829 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305831 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305834 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305836 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305839 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305841 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305844 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305846 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.308821 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305849 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305851 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305854 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305856 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305858 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305861 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305863 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.305866 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306264 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306269 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306272 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306278 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306282 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306285 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306287 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306290 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306293 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306295 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306298 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306301 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.309436 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306303 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306306 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306308 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306311 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306313 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306316 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306318 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306321 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306324 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306326 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306329 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306331 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306334 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306336 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306339 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306342 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306344 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306346 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306349 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.309918 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306353 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306358 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306361 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306363 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306367 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306370 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306373 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306375 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306378 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306381 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306383 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306386 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306389 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306392 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306394 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306397 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306399 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306402 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306404 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306407 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.310395 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306410 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306412 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306415 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306418 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306420 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306423 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306425 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306428 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306430 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306433 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306435 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306438 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306441 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306445 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306447 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306450 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306454 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306456 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306459 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306469 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.310906 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306471 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306476 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306479 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306482 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306485 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306488 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306491 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306493 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306496 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306498 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306501 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306504 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306507 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306509 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306512 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306514 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306517 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306519 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306522 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.311402 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306524 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306527 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306529 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306532 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306534 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306536 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306539 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306542 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.306546 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306618 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306628 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306640 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306644 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306653 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306656 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306660 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306664 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306667 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306670 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306673 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306677 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306680 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 00:17:08.311850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306682 2572 flags.go:64] FLAG: --cgroup-root="" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306685 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306688 2572 flags.go:64] FLAG: --client-ca-file="" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306691 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306694 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306700 2572 flags.go:64] FLAG: --cluster-domain="" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306703 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306706 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306709 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306712 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306716 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306719 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306722 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306725 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306728 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306731 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306734 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306737 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306740 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306744 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306747 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306750 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306753 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306756 2572 flags.go:64] FLAG: --enable-server="true" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306762 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 00:17:08.312374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306766 2572 flags.go:64] FLAG: --event-burst="100" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306769 2572 flags.go:64] FLAG: --event-qps="50" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306772 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306775 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306778 2572 flags.go:64] FLAG: --eviction-hard="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306784 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306787 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306789 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306792 2572 flags.go:64] FLAG: --eviction-soft="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306795 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306798 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306801 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306804 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306807 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306810 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306813 2572 flags.go:64] FLAG: --feature-gates="" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306816 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306819 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306822 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306825 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306828 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306831 2572 flags.go:64] FLAG: --help="false" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306834 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306837 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 00:17:08.312974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306840 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306843 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306847 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306850 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306853 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306856 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306859 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306862 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306865 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306868 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306871 2572 flags.go:64] FLAG: --kube-reserved="" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306874 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306876 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306879 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306882 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306884 2572 flags.go:64] FLAG: --lock-file="" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306887 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306890 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306892 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306898 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306901 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306903 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306906 2572 flags.go:64] FLAG: --logging-format="text" Jul 10 00:17:08.313571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306909 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306912 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306915 2572 flags.go:64] FLAG: --manifest-url="" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306918 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306922 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306925 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306929 2572 flags.go:64] FLAG: --max-pods="110" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306932 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306934 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306937 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306941 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306944 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306961 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306964 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306971 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306974 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306977 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306980 2572 flags.go:64] FLAG: --pod-cidr="" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306984 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306989 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306992 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306995 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.306998 2572 flags.go:64] FLAG: --port="10250" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307001 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 00:17:08.314169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307004 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0545091b452f7b5a0" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307007 2572 flags.go:64] FLAG: --qos-reserved="" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307010 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307013 2572 flags.go:64] FLAG: --register-node="true" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307016 2572 flags.go:64] FLAG: --register-with-taints="" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307020 2572 flags.go:64] FLAG: --registry-burst="10" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307023 2572 flags.go:64] FLAG: --registry-qps="5" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307025 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307028 2572 flags.go:64] FLAG: --reserved-memory="" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307031 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307034 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307037 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307040 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307043 2572 flags.go:64] FLAG: --runonce="false" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307046 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307049 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307053 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307056 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307059 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307063 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307066 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307069 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307072 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307074 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307077 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307080 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 00:17:08.314763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307083 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307086 2572 flags.go:64] FLAG: --system-cgroups="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307089 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307094 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307097 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307099 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307103 2572 flags.go:64] FLAG: --tls-min-version="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307106 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307108 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307111 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307114 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307117 2572 flags.go:64] FLAG: --v="2" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307120 2572 flags.go:64] FLAG: --version="false" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307124 2572 flags.go:64] FLAG: --vmodule="" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307128 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.307131 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307277 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307281 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307284 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307288 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307292 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307296 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307301 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.315430 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307303 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307306 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307310 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307313 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307316 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307320 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307323 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307326 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307328 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307331 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307333 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307337 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307339 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307342 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307345 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307347 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307350 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307352 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307355 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.315988 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307357 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307360 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307362 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307365 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307367 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307370 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307372 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307375 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307377 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307379 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307394 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307397 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307401 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307404 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307406 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307410 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307413 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307416 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307419 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307421 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.316439 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307424 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307426 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307429 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307432 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307434 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307437 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307440 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307442 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307445 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307447 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307450 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307452 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307455 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307458 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307462 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307465 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307467 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307470 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307472 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307475 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.316914 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307477 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307480 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307483 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307485 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307489 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307491 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307493 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307497 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307500 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307503 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307505 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307508 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307510 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307513 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307515 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307518 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307520 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307523 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307526 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.317413 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307528 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307531 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307533 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307536 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307538 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307541 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307543 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307546 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307548 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307551 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307553 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307556 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307558 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.307561 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.308320 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:17:08.317867 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.315396 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.315410 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.316342 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.316350 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316406 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316412 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316415 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316418 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316421 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316424 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316426 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316429 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316432 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316435 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316437 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316440 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316446 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316449 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316453 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.318321 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316456 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316459 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316463 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316466 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316469 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316472 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316475 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316478 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316481 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316483 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316486 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316490 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316494 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316497 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316501 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316504 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316507 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316510 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316513 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.318772 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316516 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316518 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316521 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316523 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316526 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316528 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316530 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316533 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316536 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316538 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316541 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316543 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316546 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316550 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316554 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316557 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316560 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316563 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316566 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.319251 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316569 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316572 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316576 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316578 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316581 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316584 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316587 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316589 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316592 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316595 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316597 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316600 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316602 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316604 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316607 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316609 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316612 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316614 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316617 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316619 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.319724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316622 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316625 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316628 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316631 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316633 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316636 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316639 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316641 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316644 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316646 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316649 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316651 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316654 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316657 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316659 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316661 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316664 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316666 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316669 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316671 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.320224 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316674 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316677 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316680 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316682 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316685 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316687 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.316693 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316799 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316805 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316808 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316811 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316814 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316817 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316820 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316823 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:17:08.320695 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316825 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316829 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316831 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316834 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316837 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316839 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316842 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316844 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316847 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316849 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316852 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316856 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316859 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316862 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316864 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316867 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316870 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316872 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316875 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:17:08.321072 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316878 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316881 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316883 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316886 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316890 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316893 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316896 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316899 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316901 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316904 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316907 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316909 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316912 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316914 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316917 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316920 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316922 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316925 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316928 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316930 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:17:08.321525 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316933 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316935 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316938 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316940 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316942 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316959 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316962 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316965 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316967 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316970 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316972 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316975 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316978 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316980 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316983 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316986 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316988 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316991 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316993 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:17:08.322011 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316996 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.316998 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317001 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317005 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317008 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317011 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317014 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317016 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317019 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317022 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317024 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317027 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317030 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317033 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317035 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317038 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317040 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317043 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317045 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:17:08.322448 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317047 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317050 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317053 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317055 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317057 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317060 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317063 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317065 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317068 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317070 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317073 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317075 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317078 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:08.317081 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.317086 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:17:08.322898 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.317181 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 00:17:08.323311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.319557 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 00:17:08.323311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.321154 2572 server.go:1019] "Starting client certificate rotation" Jul 10 00:17:08.323311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.321255 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 00:17:08.323311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.321326 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 00:17:08.323990 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.323975 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 00:17:08.327114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.327092 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 00:17:08.337692 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.337674 2572 log.go:25] "Validated CRI v1 runtime API" Jul 10 00:17:08.340211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.340196 2572 log.go:25] "Validated CRI v1 image API" Jul 10 00:17:08.341229 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.341201 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 00:17:08.345705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.345686 2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9fa9450f-7062-407e-8ec1-943c0e10a209:/dev/nvme0n1p4 ef933b89-bd19-4cd0-af9e-4f5743396deb:/dev/nvme0n1p3] Jul 10 00:17:08.345760 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.345705 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 00:17:08.352440 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.352328 2572 manager.go:216] Machine: {Timestamp:2026-07-10 00:17:08.350740956 +0000 UTC m=+0.377219657 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105129 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2574500c200eee76e2d700c13f390e SystemUUID:ec257450-0c20-0eee-76e2-d700c13f390e BootID:bcd9e0da-0f8e-4785-8e89-15f41f715036 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:20:0d:ed:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:20:0d:ed:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:3b:a9:b7:89:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 00:17:08.352440 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.352438 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 00:17:08.352553 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.352523 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 00:17:08.355989 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.355962 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 00:17:08.356135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.355994 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 00:17:08.356505 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.356490 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 00:17:08.357663 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.357652 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 00:17:08.357704 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.357666 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 00:17:08.357704 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.357675 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 00:17:08.358502 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.358491 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 00:17:08.359727 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.359717 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 00:17:08.364278 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364267 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 10 00:17:08.364400 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364391 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 00:17:08.364440 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364430 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 10 00:17:08.364469 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364455 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 00:17:08.364495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364475 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 00:17:08.364495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364489 2572 kubelet.go:398] "Adding apiserver pod source" Jul 10 00:17:08.364543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.364501 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 00:17:08.365040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.365024 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 00:17:08.365089 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.365054 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 00:17:08.367505 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.367488 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 00:17:08.369873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.369858 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 00:17:08.369929 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.369888 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 00:17:08.370718 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370708 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 00:17:08.370771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370724 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 00:17:08.370771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370734 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 00:17:08.370771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370742 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 00:17:08.370771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370753 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 00:17:08.370771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370759 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370809 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370817 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370826 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370831 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370839 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 00:17:08.370907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.370846 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 00:17:08.371784 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.371772 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 00:17:08.371834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.371784 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 00:17:08.375237 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.375222 2572 server.go:1268] "Started kubelet" Jul 10 00:17:08.378624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.378423 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 00:17:08.378624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.378517 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 00:17:08.378624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.378366 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 00:17:08.379180 ip-10-0-142-23 systemd[1]: Started Kubernetes Kubelet. Jul 10 00:17:08.382454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.381410 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 00:17:08.382454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.381420 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 00:17:08.382454 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.381752 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 00:17:08.382454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.381797 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 10 00:17:08.384576 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.384550 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 00:17:08.386887 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.386868 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 00:17:08.390997 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.390985 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 00:17:08.391071 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.391006 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 00:17:08.391177 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.391161 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.391902 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.391889 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 00:17:08.391995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.391906 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 00:17:08.391995 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.390800 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-23.ec2.internal.18c0c585368c2295 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-23.ec2.internal,UID:ip-10-0-142-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-23.ec2.internal,},FirstTimestamp:2026-07-10 00:17:08.375192213 +0000 UTC m=+0.401670909,LastTimestamp:2026-07-10 00:17:08.375192213 +0000 UTC m=+0.401670909,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-23.ec2.internal,}" Jul 10 00:17:08.392127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392058 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 10 00:17:08.392127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392066 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 10 00:17:08.392127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392110 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 00:17:08.392374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392323 2572 factory.go:55] Registering systemd factory Jul 10 00:17:08.392374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392345 2572 factory.go:223] Registration of the systemd container factory successfully Jul 10 00:17:08.392588 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392569 2572 factory.go:153] Registering CRI-O factory Jul 10 00:17:08.392635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392590 2572 factory.go:223] Registration of the crio container factory successfully Jul 10 00:17:08.392698 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392638 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 00:17:08.392698 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392663 2572 factory.go:103] Registering Raw factory Jul 10 00:17:08.392698 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.392679 2572 manager.go:1196] Started watching for new ooms in manager Jul 10 00:17:08.393352 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.393336 2572 manager.go:349] Starting recovery of all containers Jul 10 00:17:08.396637 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.396613 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vk6jc" Jul 10 00:17:08.398055 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.398023 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 00:17:08.398176 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.398154 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 00:17:08.403548 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.403392 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vk6jc" Jul 10 00:17:08.403628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.403482 2572 manager.go:354] Recovery completed Jul 10 00:17:08.408038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.408023 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.409781 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.409765 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.409837 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.409794 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.409837 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.409809 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.410289 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.410271 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 00:17:08.410289 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.410289 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 00:17:08.410400 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.410306 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 10 00:17:08.410400 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.410281 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.411424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.411404 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.411516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.411436 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.411516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.411451 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.412027 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.411918 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-23.ec2.internal.18c0c585389be8c7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-23.ec2.internal,UID:ip-10-0-142-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-23.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-23.ec2.internal,},FirstTimestamp:2026-07-10 00:17:08.409780423 +0000 UTC m=+0.436259121,LastTimestamp:2026-07-10 00:17:08.409780423 +0000 UTC m=+0.436259121,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-23.ec2.internal,}" Jul 10 00:17:08.412514 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.412502 2572 policy_none.go:49] "None policy: Start" Jul 10 00:17:08.412578 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.412518 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 00:17:08.412578 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.412529 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 00:17:08.414513 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.414501 2572 policy_none.go:47] "Start" Jul 10 00:17:08.457739 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.457661 2572 manager.go:337] "Starting Device Plugin manager" Jul 10 00:17:08.457739 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.457693 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 00:17:08.457739 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.457705 2572 server.go:87] "Starting device plugin registration server" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.457990 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.458002 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.458082 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.458166 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.458176 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.458670 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 00:17:08.478471 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.458709 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.505387 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.505355 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 00:17:08.506699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.506674 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 00:17:08.506779 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.506724 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 00:17:08.506779 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.506749 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 00:17:08.506878 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.506787 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 00:17:08.508745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.508728 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:17:08.558791 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.558742 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.559683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.559666 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.559741 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.559698 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.559741 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.559711 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.559741 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.559739 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.571068 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.571054 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.571124 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.571076 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-23.ec2.internal\": node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.590845 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.590824 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.607603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.607575 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal"] Jul 10 00:17:08.607669 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.607645 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.609203 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.609191 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.609271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.609227 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.609271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.609238 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.611618 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.611604 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.611753 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.611740 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.611789 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.611767 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.612363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612347 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.612431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612375 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.612431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612386 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.612431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612398 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.612431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612419 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.612431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.612428 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.614662 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.614649 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.614709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.614673 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:17:08.615369 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.615351 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:17:08.615449 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.615381 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:17:08.615449 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.615396 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:17:08.647551 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.647535 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-23.ec2.internal\" not found" node="ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.651801 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.651777 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-23.ec2.internal\" not found" node="ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.691750 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.691737 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.692981 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.692965 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.693036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.692989 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.693036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.693006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b630ef5ac97d738b7ebb71a3cf093891-config\") pod \"kube-apiserver-proxy-ip-10-0-142-23.ec2.internal\" (UID: \"b630ef5ac97d738b7ebb71a3cf093891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.792105 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.792078 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.793237 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793216 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.793316 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793253 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.793316 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b630ef5ac97d738b7ebb71a3cf093891-config\") pod \"kube-apiserver-proxy-ip-10-0-142-23.ec2.internal\" (UID: \"b630ef5ac97d738b7ebb71a3cf093891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.793418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793331 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b630ef5ac97d738b7ebb71a3cf093891-config\") pod \"kube-apiserver-proxy-ip-10-0-142-23.ec2.internal\" (UID: \"b630ef5ac97d738b7ebb71a3cf093891\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.793418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793321 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.793418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.793345 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fda362feaa8370fcc5c00f7c6fa9714b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal\" (UID: \"fda362feaa8370fcc5c00f7c6fa9714b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.892593 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.892516 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:08.949821 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.949789 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.955115 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:08.954552 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:08.993131 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:08.993097 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.093524 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.093492 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.194011 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.193920 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.294666 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.294635 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.320882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.320862 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 00:17:09.321417 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.321037 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 00:17:09.395038 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.395012 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.410121 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.410097 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 00:12:09 +0000 UTC" deadline="2028-04-13 12:54:51.129557332 +0000 UTC" Jul 10 00:17:09.410121 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.410121 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15444h37m41.719439207s" Jul 10 00:17:09.495714 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.495677 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.520691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.520666 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:17:09.593283 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.593259 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:17:09.594545 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:09.594509 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb630ef5ac97d738b7ebb71a3cf093891.slice/crio-62049bc0481b1a05af2d6a714188bb73a466b900982db7b6b14829ded969fcba WatchSource:0}: Error finding container 62049bc0481b1a05af2d6a714188bb73a466b900982db7b6b14829ded969fcba: Status 404 returned error can't find the container with id 62049bc0481b1a05af2d6a714188bb73a466b900982db7b6b14829ded969fcba Jul 10 00:17:09.594774 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:09.594755 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda362feaa8370fcc5c00f7c6fa9714b.slice/crio-57d5f3e0f4fa507cd3aceef3060a9ef79026c299cdf0f2adf7f1a6a9b9346ebe WatchSource:0}: Error finding container 57d5f3e0f4fa507cd3aceef3060a9ef79026c299cdf0f2adf7f1a6a9b9346ebe: Status 404 returned error can't find the container with id 57d5f3e0f4fa507cd3aceef3060a9ef79026c299cdf0f2adf7f1a6a9b9346ebe Jul 10 00:17:09.596093 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:09.596078 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-23.ec2.internal\" not found" Jul 10 00:17:09.598680 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.598667 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:17:09.627320 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.627298 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:17:09.691407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.691373 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" Jul 10 00:17:09.700908 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.700882 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 00:17:09.701873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.701858 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" Jul 10 00:17:09.714650 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:09.714631 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 00:17:10.254757 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.254730 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:17:10.365870 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.365830 2572 apiserver.go:52] "Watching apiserver" Jul 10 00:17:10.374509 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.374479 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 00:17:10.374897 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.374870 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d","openshift-cluster-node-tuning-operator/tuned-6frs4","openshift-image-registry/node-ca-dmbpk","openshift-multus/network-metrics-daemon-n7czp","openshift-network-operator/iptables-alerter-6cj7p","kube-system/global-pull-secret-syncer-2c8lv","openshift-dns/node-resolver-v8t9f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal","openshift-multus/multus-additional-cni-plugins-wb9hb","openshift-multus/multus-r9kxr","openshift-network-diagnostics/network-check-target-ql7b9","openshift-ovn-kubernetes/ovnkube-node-prt82","kube-system/konnectivity-agent-8hbxb"] Jul 10 00:17:10.377576 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.377554 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.377686 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.377645 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:10.380074 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.380053 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.382436 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.382387 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 00:17:10.382780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.382531 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.382881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.382848 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.383498 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.383474 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 00:17:10.383604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.383499 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 00:17:10.383604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.383579 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 00:17:10.383715 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.383697 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qjs5d\"" Jul 10 00:17:10.383768 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.383753 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.385327 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.385301 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.386910 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.386894 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.387008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.386892 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.387578 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387556 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 00:17:10.387663 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387584 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ns7hv\"" Jul 10 00:17:10.387727 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387672 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.388007 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387807 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.388007 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387866 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cnzl5\"" Jul 10 00:17:10.388007 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.387955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.388236 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.388041 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:10.390136 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.390116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.392002 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.391969 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 00:17:10.392260 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.392216 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 00:17:10.392884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.392723 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.392884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.392753 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.392884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.392786 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wf4s2\"" Jul 10 00:17:10.392884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.392797 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.395342 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.395412 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.395674 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.395710 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z5t2p\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.398114 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.398414 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.399403 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.398609 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lbvjh\"" Jul 10 00:17:10.399864 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.399600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401004 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-serviceca\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401045 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-env-overrides\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysconfig\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-var-lib-kubelet\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401105 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-konnectivity-ca\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401121 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-kubelet\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401137 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-node-log\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401176 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd5b2\" (UniqueName: \"kubernetes.io/projected/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-kube-api-access-sd5b2\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401247 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-dbus\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401277 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p76\" (UniqueName: \"kubernetes.io/projected/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-kube-api-access-z7p76\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401307 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-systemd-units\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401331 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-netns\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401355 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-systemd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401379 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-etc-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f623af4-76b4-464c-96ef-5e17791307a8-ovn-node-metrics-cert\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401428 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-modprobe-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-bin\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401473 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-netd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401497 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-config\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-systemd\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401544 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-run\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-iptables-alerter-script\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401593 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.401649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401625 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-lib-modules\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-tuned\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401707 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401733 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401754 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401763 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-host\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401787 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401810 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfml\" (UniqueName: \"kubernetes.io/projected/13e20826-95ee-455c-8405-f23540349d1e-kube-api-access-8nfml\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401834 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-var-lib-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401846 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401859 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-log-socket\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-host\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401914 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-kubelet-config\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401939 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-ovn\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401963 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhrgf\" (UniqueName: \"kubernetes.io/projected/5f623af4-76b4-464c-96ef-5e17791307a8-kube-api-access-xhrgf\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-conf\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.401982 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 00:17:10.402404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402014 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-host-slash\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402068 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sstw5\" (UniqueName: \"kubernetes.io/projected/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-kube-api-access-sstw5\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402128 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-agent-certs\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402183 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9dtgm\"" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402200 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-slash\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402263 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-script-lib\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-kubernetes\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-sys\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.402994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.402354 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-tmp\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.403424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.403404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.405620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.405597 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 00:17:10.405814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.405791 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8c825\"" Jul 10 00:17:10.405814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.405813 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 00:17:10.405956 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.405896 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 00:17:10.406120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.406104 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 00:17:10.406120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.406118 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 00:17:10.406236 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.406213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.408656 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.408302 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 00:17:10.408656 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.408359 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4pdxd\"" Jul 10 00:17:10.408656 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.408414 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:10.408656 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.408472 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:10.423396 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.423349 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nchn7" Jul 10 00:17:10.430888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.430863 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nchn7" Jul 10 00:17:10.493340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.493309 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 00:17:10.503077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.503227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-multus\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.503227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-tmp-dir\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.503227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503176 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.503272 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-ovn\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.503364 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:11.003328784 +0000 UTC m=+3.029807483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhrgf\" (UniqueName: \"kubernetes.io/projected/5f623af4-76b4-464c-96ef-5e17791307a8-kube-api-access-xhrgf\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503394 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-ovn\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503421 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-agent-certs\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.503501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gtfc\" (UniqueName: \"kubernetes.io/projected/623f5d31-40e4-4687-af26-0c4a575523be-kube-api-access-6gtfc\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxmxp\" (UniqueName: \"kubernetes.io/projected/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-kube-api-access-hxmxp\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503575 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-serviceca\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-var-lib-kubelet\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503628 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503708 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-kubelet\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503732 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-os-release\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503731 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-var-lib-kubelet\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503761 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-node-log\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503809 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-node-log\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503849 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-device-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503847 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503894 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.503944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503936 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p76\" (UniqueName: \"kubernetes.io/projected/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-kube-api-access-z7p76\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.503986 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f623af4-76b4-464c-96ef-5e17791307a8-ovn-node-metrics-cert\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-modprobe-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504051 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-system-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504068 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-serviceca\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504182 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-modprobe-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504242 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-netns\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504275 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-netd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-systemd\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504360 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-run\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504390 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-netd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504391 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504446 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-systemd\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-run\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504465 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-socket-dir-parent\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.504603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-conf-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504492 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504517 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504546 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sstw5\" (UniqueName: \"kubernetes.io/projected/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-kube-api-access-sstw5\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504581 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-d\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504599 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-host\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfml\" (UniqueName: \"kubernetes.io/projected/13e20826-95ee-455c-8405-f23540349d1e-kube-api-access-8nfml\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-var-lib-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-log-socket\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-host\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndks5\" (UniqueName: \"kubernetes.io/projected/04129ab2-4456-4acb-839a-b296be7fa6f5-kube-api-access-ndks5\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.504755 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.504807 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:11.00479202 +0000 UTC m=+3.031270707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-kubelet-config\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504808 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-kubelet-config\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.505479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-var-lib-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504848 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-log-socket\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504873 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-conf\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504880 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-host\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504899 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-host-slash\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504925 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-host\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504928 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-registration-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504962 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-os-release\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.504980 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-host-slash\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505008 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-etc-kubernetes\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysctl-conf\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505034 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-hosts-file\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505067 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-slash\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505094 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-script-lib\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-kubernetes\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505156 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-sys\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505172 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.506384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-tmp\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505205 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-env-overrides\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505234 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysconfig\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505244 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-slash\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505260 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-konnectivity-ca\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505311 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-kubernetes\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505336 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-cni-binary-copy\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505370 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-sys\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505370 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-k8s-cni-cncf-io\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505409 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-multus-daemon-config\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-system-cni-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505435 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-sysconfig\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-kubelet\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd5b2\" (UniqueName: \"kubernetes.io/projected/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-kube-api-access-sd5b2\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505785 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-cnibin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505827 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nswph\" (UniqueName: \"kubernetes.io/projected/45825668-2a70-4327-a7f5-6ad932ab0eb6-kube-api-access-nswph\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.507193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505870 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-dbus\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505906 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-systemd-units\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505931 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-netns\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.505964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-systemd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506021 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-etc-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-socket-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506089 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-sys-fs\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-bin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506125 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-script-lib\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-bin\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506214 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-cni-bin\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506379 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-kubelet\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506655 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-env-overrides\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506739 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-etc-openvswitch\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506767 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-konnectivity-ca\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506784 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-host-run-netns\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506852 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-run-systemd\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.507834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506866 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2168193-a602-41e7-adc0-7c0a76072865-dbus\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506919 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5f623af4-76b4-464c-96ef-5e17791307a8-systemd-units\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-config\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.506992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-iptables-alerter-script\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507025 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-hostroot\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-multus-certs\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-cnibin\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507122 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-lib-modules\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-tuned\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507197 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507768 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5f623af4-76b4-464c-96ef-5e17791307a8-ovnkube-config\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.507872 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-iptables-alerter-script\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.508717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.508002 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-lib-modules\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.513261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.509556 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f623af4-76b4-464c-96ef-5e17791307a8-ovn-node-metrics-cert\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.513261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.509987 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-etc-tuned\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.516074 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.516028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhrgf\" (UniqueName: \"kubernetes.io/projected/5f623af4-76b4-464c-96ef-5e17791307a8-kube-api-access-xhrgf\") pod \"ovnkube-node-prt82\" (UID: \"5f623af4-76b4-464c-96ef-5e17791307a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.516697 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.516672 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p76\" (UniqueName: \"kubernetes.io/projected/96c8aa3a-d36e-42fd-b36f-54a04b7d24d4-kube-api-access-z7p76\") pod \"node-ca-dmbpk\" (UID: \"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4\") " pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.516809 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.516795 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd5b2\" (UniqueName: \"kubernetes.io/projected/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-kube-api-access-sd5b2\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.517831 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.517808 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfml\" (UniqueName: \"kubernetes.io/projected/13e20826-95ee-455c-8405-f23540349d1e-kube-api-access-8nfml\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:10.517982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.517962 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9-agent-certs\") pod \"konnectivity-agent-8hbxb\" (UID: \"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9\") " pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.519388 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.518865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" event={"ID":"fda362feaa8370fcc5c00f7c6fa9714b","Type":"ContainerStarted","Data":"57d5f3e0f4fa507cd3aceef3060a9ef79026c299cdf0f2adf7f1a6a9b9346ebe"} Jul 10 00:17:10.519388 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.519175 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sstw5\" (UniqueName: \"kubernetes.io/projected/e98dccb8-07d7-4adc-9203-c7cd0dd63b45-kube-api-access-sstw5\") pod \"iptables-alerter-6cj7p\" (UID: \"e98dccb8-07d7-4adc-9203-c7cd0dd63b45\") " pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.520389 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.520366 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" event={"ID":"b630ef5ac97d738b7ebb71a3cf093891","Type":"ContainerStarted","Data":"62049bc0481b1a05af2d6a714188bb73a466b900982db7b6b14829ded969fcba"} Jul 10 00:17:10.520471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.520442 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0-tmp\") pod \"tuned-6frs4\" (UID: \"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0\") " pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.608223 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-socket-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608232 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-sys-fs\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608256 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-bin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608279 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-hostroot\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-multus-certs\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-cnibin\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.608408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-multus\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608432 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-tmp-dir\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608457 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608485 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gtfc\" (UniqueName: \"kubernetes.io/projected/623f5d31-40e4-4687-af26-0c4a575523be-kube-api-access-6gtfc\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxmxp\" (UniqueName: \"kubernetes.io/projected/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-kube-api-access-hxmxp\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608536 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-kubelet\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-os-release\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-device-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608639 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608670 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-system-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-netns\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608734 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-socket-dir-parent\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.608829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608793 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-conf-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608832 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608888 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndks5\" (UniqueName: \"kubernetes.io/projected/04129ab2-4456-4acb-839a-b296be7fa6f5-kube-api-access-ndks5\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608922 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-registration-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608948 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-os-release\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.608976 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-etc-kubernetes\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609005 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-hosts-file\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-cni-binary-copy\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609138 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-k8s-cni-cncf-io\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609167 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-multus-daemon-config\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-system-cni-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-cnibin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.609543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609250 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nswph\" (UniqueName: \"kubernetes.io/projected/45825668-2a70-4327-a7f5-6ad932ab0eb6-kube-api-access-nswph\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609693 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-socket-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609772 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-sys-fs\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609816 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-bin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609855 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-hostroot\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609894 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-multus-certs\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.609934 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-cnibin\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610127 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.610197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610193 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.610600 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610234 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-cni-multus\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.610600 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610472 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-tmp-dir\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.610600 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610592 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.610818 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610632 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.610818 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610660 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.610818 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610675 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-socket-dir-parent\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.610818 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610726 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-conf-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610854 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-multus-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610883 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-var-lib-kubelet\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610938 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-os-release\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.611059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.610973 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-device-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.611273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611109 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04129ab2-4456-4acb-839a-b296be7fa6f5-registration-dir\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.611273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611178 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-os-release\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611230 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-etc-kubernetes\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611342 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-hosts-file\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.611432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611367 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45825668-2a70-4327-a7f5-6ad932ab0eb6-cni-binary-copy\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.611432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611416 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-system-cni-dir\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611446 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-netns\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611476 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45825668-2a70-4327-a7f5-6ad932ab0eb6-system-cni-dir\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.611590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611503 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-host-run-k8s-cni-cncf-io\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611535 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/623f5d31-40e4-4687-af26-0c4a575523be-cnibin\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611556 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-cni-binary-copy\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.611782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.611707 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/623f5d31-40e4-4687-af26-0c4a575523be-multus-daemon-config\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.617864 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.617813 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:10.617864 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.617854 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:10.617864 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.617869 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:10.618097 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:10.617930 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:11.117912257 +0000 UTC m=+3.144390951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:10.621656 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.621628 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndks5\" (UniqueName: \"kubernetes.io/projected/04129ab2-4456-4acb-839a-b296be7fa6f5-kube-api-access-ndks5\") pod \"aws-ebs-csi-driver-node-tsx2d\" (UID: \"04129ab2-4456-4acb-839a-b296be7fa6f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.622214 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.622087 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nswph\" (UniqueName: \"kubernetes.io/projected/45825668-2a70-4327-a7f5-6ad932ab0eb6-kube-api-access-nswph\") pod \"multus-additional-cni-plugins-wb9hb\" (UID: \"45825668-2a70-4327-a7f5-6ad932ab0eb6\") " pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.623867 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.623844 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxmxp\" (UniqueName: \"kubernetes.io/projected/5fe2c04e-788c-49c7-bdb0-40d66299f6f9-kube-api-access-hxmxp\") pod \"node-resolver-v8t9f\" (UID: \"5fe2c04e-788c-49c7-bdb0-40d66299f6f9\") " pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.624282 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.624242 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gtfc\" (UniqueName: \"kubernetes.io/projected/623f5d31-40e4-4687-af26-0c4a575523be-kube-api-access-6gtfc\") pod \"multus-r9kxr\" (UID: \"623f5d31-40e4-4687-af26-0c4a575523be\") " pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.693393 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.693308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:10.700092 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.700070 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6frs4" Jul 10 00:17:10.705344 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.705310 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f623af4_76b4_464c_96ef_5e17791307a8.slice/crio-192510637e4d353f4961879360740672a5fed673992da8e1544b90ee9ae334d3 WatchSource:0}: Error finding container 192510637e4d353f4961879360740672a5fed673992da8e1544b90ee9ae334d3: Status 404 returned error can't find the container with id 192510637e4d353f4961879360740672a5fed673992da8e1544b90ee9ae334d3 Jul 10 00:17:10.710903 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.710873 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dmbpk" Jul 10 00:17:10.711219 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.711194 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9c55b6_4dab_4e0b_9a9c_59b7c7c040d0.slice/crio-55c5233328652d92c509b6940f8bed669a969dc756b809ddc9358437488727c3 WatchSource:0}: Error finding container 55c5233328652d92c509b6940f8bed669a969dc756b809ddc9358437488727c3: Status 404 returned error can't find the container with id 55c5233328652d92c509b6940f8bed669a969dc756b809ddc9358437488727c3 Jul 10 00:17:10.716975 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.716953 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6cj7p" Jul 10 00:17:10.723535 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.723285 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c8aa3a_d36e_42fd_b36f_54a04b7d24d4.slice/crio-3ab01cf31a8cb516079e5fad3c9b161140072cccb37ccd08eb55b2894ed273ad WatchSource:0}: Error finding container 3ab01cf31a8cb516079e5fad3c9b161140072cccb37ccd08eb55b2894ed273ad: Status 404 returned error can't find the container with id 3ab01cf31a8cb516079e5fad3c9b161140072cccb37ccd08eb55b2894ed273ad Jul 10 00:17:10.724989 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.724641 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:10.728174 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.728139 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98dccb8_07d7_4adc_9203_c7cd0dd63b45.slice/crio-df14ec25a71ab2e435d53c3d97dcf8429f27f2a66bee1c985f488675f4d38374 WatchSource:0}: Error finding container df14ec25a71ab2e435d53c3d97dcf8429f27f2a66bee1c985f488675f4d38374: Status 404 returned error can't find the container with id df14ec25a71ab2e435d53c3d97dcf8429f27f2a66bee1c985f488675f4d38374 Jul 10 00:17:10.733741 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.733717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v8t9f" Jul 10 00:17:10.737737 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.737714 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d84ec5b_9b0b_4e1f_a21e_e8240ba459e9.slice/crio-f22b8b7af5feb661eadd449bec5f614d36daf09291205530fe548674465ea243 WatchSource:0}: Error finding container f22b8b7af5feb661eadd449bec5f614d36daf09291205530fe548674465ea243: Status 404 returned error can't find the container with id f22b8b7af5feb661eadd449bec5f614d36daf09291205530fe548674465ea243 Jul 10 00:17:10.746395 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.746167 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" Jul 10 00:17:10.746996 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.746914 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe2c04e_788c_49c7_bdb0_40d66299f6f9.slice/crio-a4c1d8b473c47be9389dfbcf8803bdf8161005edff97ae988411be915e55ccfe WatchSource:0}: Error finding container a4c1d8b473c47be9389dfbcf8803bdf8161005edff97ae988411be915e55ccfe: Status 404 returned error can't find the container with id a4c1d8b473c47be9389dfbcf8803bdf8161005edff97ae988411be915e55ccfe Jul 10 00:17:10.755456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.755431 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" Jul 10 00:17:10.760494 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.760469 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04129ab2_4456_4acb_839a_b296be7fa6f5.slice/crio-f86ea1d0a990f8514586fe69ad51e52113ac1d603229fd20f101138b7bdde82b WatchSource:0}: Error finding container f86ea1d0a990f8514586fe69ad51e52113ac1d603229fd20f101138b7bdde82b: Status 404 returned error can't find the container with id f86ea1d0a990f8514586fe69ad51e52113ac1d603229fd20f101138b7bdde82b Jul 10 00:17:10.763440 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:10.763210 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r9kxr" Jul 10 00:17:10.768441 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.768413 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45825668_2a70_4327_a7f5_6ad932ab0eb6.slice/crio-997605728b88235353ca553b737e13e60ece9e1a82835a44899a26a8e188aa85 WatchSource:0}: Error finding container 997605728b88235353ca553b737e13e60ece9e1a82835a44899a26a8e188aa85: Status 404 returned error can't find the container with id 997605728b88235353ca553b737e13e60ece9e1a82835a44899a26a8e188aa85 Jul 10 00:17:10.778724 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:10.778699 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623f5d31_40e4_4687_af26_0c4a575523be.slice/crio-f13d260be8f3533e812e193e5f099529736624b0f25d6c6185f46965f79b4106 WatchSource:0}: Error finding container f13d260be8f3533e812e193e5f099529736624b0f25d6c6185f46965f79b4106: Status 404 returned error can't find the container with id f13d260be8f3533e812e193e5f099529736624b0f25d6c6185f46965f79b4106 Jul 10 00:17:11.012550 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.012510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:11.012751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.012585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:11.012751 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.012664 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:11.012751 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.012683 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:11.012751 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.012731 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:12.012715867 +0000 UTC m=+4.039194569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:11.012751 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.012747 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:12.012739632 +0000 UTC m=+4.039218316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:11.214034 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.213994 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:11.214236 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.214195 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:11.214236 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.214221 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:11.214236 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.214232 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:11.214399 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:11.214302 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:12.214282144 +0000 UTC m=+4.240760828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:11.434332 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.434236 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:12:11 +0000 UTC" deadline="2027-12-27 15:00:47.628180864 +0000 UTC" Jul 10 00:17:11.434332 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.434274 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12854h43m36.193910857s" Jul 10 00:17:11.524072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.524019 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dmbpk" event={"ID":"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4","Type":"ContainerStarted","Data":"3ab01cf31a8cb516079e5fad3c9b161140072cccb37ccd08eb55b2894ed273ad"} Jul 10 00:17:11.525351 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.525317 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerStarted","Data":"997605728b88235353ca553b737e13e60ece9e1a82835a44899a26a8e188aa85"} Jul 10 00:17:11.526547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.526510 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" event={"ID":"04129ab2-4456-4acb-839a-b296be7fa6f5","Type":"ContainerStarted","Data":"f86ea1d0a990f8514586fe69ad51e52113ac1d603229fd20f101138b7bdde82b"} Jul 10 00:17:11.527830 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.527791 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v8t9f" event={"ID":"5fe2c04e-788c-49c7-bdb0-40d66299f6f9","Type":"ContainerStarted","Data":"a4c1d8b473c47be9389dfbcf8803bdf8161005edff97ae988411be915e55ccfe"} Jul 10 00:17:11.529078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.529053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cj7p" event={"ID":"e98dccb8-07d7-4adc-9203-c7cd0dd63b45","Type":"ContainerStarted","Data":"df14ec25a71ab2e435d53c3d97dcf8429f27f2a66bee1c985f488675f4d38374"} Jul 10 00:17:11.530308 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.530284 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6frs4" event={"ID":"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0","Type":"ContainerStarted","Data":"55c5233328652d92c509b6940f8bed669a969dc756b809ddc9358437488727c3"} Jul 10 00:17:11.531541 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.531512 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"192510637e4d353f4961879360740672a5fed673992da8e1544b90ee9ae334d3"} Jul 10 00:17:11.532770 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.532748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r9kxr" event={"ID":"623f5d31-40e4-4687-af26-0c4a575523be","Type":"ContainerStarted","Data":"f13d260be8f3533e812e193e5f099529736624b0f25d6c6185f46965f79b4106"} Jul 10 00:17:11.533833 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:11.533812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hbxb" event={"ID":"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9","Type":"ContainerStarted","Data":"f22b8b7af5feb661eadd449bec5f614d36daf09291205530fe548674465ea243"} Jul 10 00:17:12.021297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.021211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:12.021297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.021298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:12.021553 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.021452 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:12.021553 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.021520 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:14.021500256 +0000 UTC m=+6.047978958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:12.021982 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.021949 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:12.022059 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.022013 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:14.021995447 +0000 UTC m=+6.048474135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:12.222384 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.222342 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:12.222587 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.222509 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:12.222587 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.222528 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:12.222587 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.222538 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:12.222587 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.222587 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:14.2225732 +0000 UTC m=+6.249051885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:12.435460 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.435189 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:12:11 +0000 UTC" deadline="2028-03-29 00:54:32.505491456 +0000 UTC" Jul 10 00:17:12.435866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.435466 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15072h37m20.070033733s" Jul 10 00:17:12.507816 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.507784 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:12.507943 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.507918 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:12.508285 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.508264 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:12.508394 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.508374 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:12.508447 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.508433 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:12.508550 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:12.508530 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:12.550005 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:12.549947 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" event={"ID":"b630ef5ac97d738b7ebb71a3cf093891","Type":"ContainerStarted","Data":"495f74fb7c0e579b93b26ce165165db463f80b3892e704f53c97e80ceff82428"} Jul 10 00:17:13.565255 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:13.565217 2572 generic.go:356] "Generic (PLEG): container finished" podID="fda362feaa8370fcc5c00f7c6fa9714b" containerID="50f64502777c66f27c227c5452290ef4d8ed86c02e30678f0c105549489c4ce4" exitCode=0 Jul 10 00:17:13.565772 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:13.565417 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" event={"ID":"fda362feaa8370fcc5c00f7c6fa9714b","Type":"ContainerDied","Data":"50f64502777c66f27c227c5452290ef4d8ed86c02e30678f0c105549489c4ce4"} Jul 10 00:17:13.581672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:13.581619 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-23.ec2.internal" podStartSLOduration=3.5815947169999998 podStartE2EDuration="3.581594717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:12.565590011 +0000 UTC m=+4.592068724" watchObservedRunningTime="2026-07-10 00:17:13.581594717 +0000 UTC m=+5.608073428" Jul 10 00:17:14.041839 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.041791 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:14.042066 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.042021 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:14.044450 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.044418 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:14.044450 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.044448 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:14.044641 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.044519 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:18.044495059 +0000 UTC m=+10.070973747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:14.044641 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.044539 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:18.044528277 +0000 UTC m=+10.071006960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:14.243858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.243752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:14.244068 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.243969 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:14.244068 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.244011 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:14.244068 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.244025 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:14.244246 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.244090 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:18.244072363 +0000 UTC m=+10.270551051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:14.508186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.508158 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:14.508433 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.508288 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:14.508710 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.508688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:14.508822 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.508797 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:14.510723 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.510574 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:14.510723 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:14.510683 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:14.573088 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:14.572914 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" event={"ID":"fda362feaa8370fcc5c00f7c6fa9714b","Type":"ContainerStarted","Data":"e7daa568e5c681a30ba8c51ed6e07e1bc0eac977c86517ccc5e53c825db3e9df"} Jul 10 00:17:16.509271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:16.509240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:16.509637 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:16.509348 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:16.509690 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:16.509634 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:16.509726 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:16.509704 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:16.509767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:16.509734 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:16.509803 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:16.509778 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:18.083623 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.083585 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:18.084133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.083678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:18.084133 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.083760 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:18.084133 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.083818 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:18.084133 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.083890 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:26.08386946 +0000 UTC m=+18.110348148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:18.084133 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.083951 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:26.083929021 +0000 UTC m=+18.110407723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:18.285325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.285287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:18.285544 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.285525 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:18.285632 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.285556 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:18.285632 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.285568 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:18.285632 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.285627 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:26.285605873 +0000 UTC m=+18.312084569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.508101 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.508101 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.508236 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:18.508260 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.508374 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:18.510006 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:18.508458 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:20.507973 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:20.507985 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:20.508103 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:20.508121 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:20.508217 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:20.508435 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:20.508309 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:22.507310 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:22.507278 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:22.507689 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:22.507395 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:22.507689 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:22.507637 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:22.507796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:22.507690 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:22.507796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:22.507729 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:22.507796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:22.507775 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:23.594588 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.594293 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hbxb" event={"ID":"7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9","Type":"ContainerStarted","Data":"02dcbe3b1f90f4cc49a13113934fdc2098d1378b0511762ff565f8fbcb7200a5"} Jul 10 00:17:23.595921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.595883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dmbpk" event={"ID":"96c8aa3a-d36e-42fd-b36f-54a04b7d24d4","Type":"ContainerStarted","Data":"fcea906fa98939dcb73ff747786cc9d09d35047fb2caeaa1feb02f495b8afdd4"} Jul 10 00:17:23.597457 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.597414 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerStarted","Data":"85c6522a4b710d129946e984bd88e8425068db443ed9a3020dc4af8aac47df62"} Jul 10 00:17:23.599021 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.598997 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" event={"ID":"04129ab2-4456-4acb-839a-b296be7fa6f5","Type":"ContainerStarted","Data":"172576885a0b6c9415a8ce34e40adfccb29657ca334879eb2f70c79f3a3420d3"} Jul 10 00:17:23.600407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.600384 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v8t9f" event={"ID":"5fe2c04e-788c-49c7-bdb0-40d66299f6f9","Type":"ContainerStarted","Data":"9351a0dd76c7352415356dfd79b89ee469863e4964b7171153b3daeeed7bf28a"} Jul 10 00:17:23.601898 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.601877 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6frs4" event={"ID":"af9c55b6-4dab-4e0b-9a9c-59b7c7c040d0","Type":"ContainerStarted","Data":"586e3d471a7eaf3294e91a4fe3153dabe063cc0452b36b5ffb9eab9987023108"} Jul 10 00:17:23.607799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.607755 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-23.ec2.internal" podStartSLOduration=13.607739183 podStartE2EDuration="13.607739183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:14.589968321 +0000 UTC m=+6.616447042" watchObservedRunningTime="2026-07-10 00:17:23.607739183 +0000 UTC m=+15.634217888" Jul 10 00:17:23.608184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.608158 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hbxb" podStartSLOduration=2.896074992 podStartE2EDuration="14.608149173s" totalImagesPullingTime="11.712074181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:23.607531317 +0000 UTC m=+15.634010023" watchObservedRunningTime="2026-07-10 00:17:23.608149173 +0000 UTC m=+15.634627876" Jul 10 00:17:23.622669 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.622619 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6frs4" podStartSLOduration=2.573355471 podStartE2EDuration="14.622606139s" totalImagesPullingTime="12.049250668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:23.622321949 +0000 UTC m=+15.648800654" watchObservedRunningTime="2026-07-10 00:17:23.622606139 +0000 UTC m=+15.649084842" Jul 10 00:17:23.639894 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.639848 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v8t9f" podStartSLOduration=2.906252898 podStartE2EDuration="14.63982924s" totalImagesPullingTime="11.733576342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:23.634808099 +0000 UTC m=+15.661286803" watchObservedRunningTime="2026-07-10 00:17:23.63982924 +0000 UTC m=+15.666307946" Jul 10 00:17:23.683023 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:23.682912 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dmbpk" podStartSLOduration=2.923064081 podStartE2EDuration="14.682892517s" totalImagesPullingTime="11.759828436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:23.682375587 +0000 UTC m=+15.708854309" watchObservedRunningTime="2026-07-10 00:17:23.682892517 +0000 UTC m=+15.709371223" Jul 10 00:17:24.508138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:24.508105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:24.508293 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:24.508147 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:24.508293 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:24.508148 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:24.508293 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:24.508233 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:24.508453 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:24.508293 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:24.508453 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:24.508407 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:24.605811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:24.605762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6cj7p" event={"ID":"e98dccb8-07d7-4adc-9203-c7cd0dd63b45","Type":"ContainerStarted","Data":"56ef06ed4e3b2a1896fdda0a6f0a12e1922b58c2922bc69638d292f4f945db79"} Jul 10 00:17:24.619777 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:24.619720 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6cj7p" podStartSLOduration=3.859840296 podStartE2EDuration="15.619701437s" totalImagesPullingTime="11.759861141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:24.61919839 +0000 UTC m=+16.645677094" watchObservedRunningTime="2026-07-10 00:17:24.619701437 +0000 UTC m=+16.646180150" Jul 10 00:17:25.660716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:25.660675 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:25.661337 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:25.661286 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:26.150506 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.150456 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:26.150693 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.150543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:26.150693 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.150656 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:26.150693 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.150669 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:26.150848 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.150739 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:17:42.150719188 +0000 UTC m=+34.177197872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:26.150848 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.150758 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:42.150749314 +0000 UTC m=+34.177227999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:26.354334 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.352053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:26.354334 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.352239 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:26.354334 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.352264 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:26.354334 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.352277 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:26.354334 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.352347 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:17:42.352327471 +0000 UTC m=+34.378806173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:26.507873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.507845 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:26.508086 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.507844 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:26.508086 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:26.507988 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:26.508086 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.507984 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:26.508086 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.508077 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:26.508349 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:26.508163 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:27.613820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:27.613740 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="85c6522a4b710d129946e984bd88e8425068db443ed9a3020dc4af8aac47df62" exitCode=0 Jul 10 00:17:27.613820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:27.613798 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"85c6522a4b710d129946e984bd88e8425068db443ed9a3020dc4af8aac47df62"} Jul 10 00:17:28.508599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:28.508384 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:28.508784 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:28.508459 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:28.508784 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:28.508683 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:28.508896 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:28.508813 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:28.508896 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:28.508487 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:28.509010 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:28.508925 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:30.507288 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:30.507255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:30.507734 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:30.507368 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:30.507734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:30.507256 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:30.507734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:30.507399 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:30.507734 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:30.507473 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:30.507734 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:30.507566 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:31.864225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:31.863811 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 00:17:32.471978 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.471611 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T00:17:31.863830662Z","UUID":"887bd9ff-2087-4016-935c-c939fea67bfa","Handler":null,"Name":"","Endpoint":""} Jul 10 00:17:32.473337 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.473313 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 00:17:32.473467 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.473359 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 00:17:32.507070 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.507043 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:32.507236 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:32.507163 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:32.507319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.507295 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:32.507319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.507312 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:32.507509 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:32.507430 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:32.507627 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:32.507533 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:32.626983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.626924 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" event={"ID":"04129ab2-4456-4acb-839a-b296be7fa6f5","Type":"ContainerStarted","Data":"1df00c24278e831dbb03e7841eb65073adcde9893179855e35c0130902ace893"} Jul 10 00:17:32.630270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630242 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:17:32.630629 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630602 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f623af4-76b4-464c-96ef-5e17791307a8" containerID="a432cbe82c9161a3310e582109587fe455a31845dba6c308cb11a056cbf4e5c3" exitCode=1 Jul 10 00:17:32.630744 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630671 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"64536f397cda211aefa749b02bf5c1ee6b2817e9564dce594910ce9dd3aa635d"} Jul 10 00:17:32.630744 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630734 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"be94f78f514e53c661d42078729ea87c2f43a1470c126d489a9b947b5b1902af"} Jul 10 00:17:32.630861 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630765 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"976f27b2ad6c60861a94455f76001138bfb9a81f277eba600476399db6818b0f"} Jul 10 00:17:32.630861 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630798 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"a888131e8672ea09e9be32edad599e28438a53ee4abe53167804df0795044afe"} Jul 10 00:17:32.630861 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630828 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerDied","Data":"a432cbe82c9161a3310e582109587fe455a31845dba6c308cb11a056cbf4e5c3"} Jul 10 00:17:32.630975 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.630864 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"cf2ca822ca1e6e20400e14a3986adbd1e4213a51456cdf7a63ca291d1bbfa017"} Jul 10 00:17:32.632676 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.632651 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r9kxr" event={"ID":"623f5d31-40e4-4687-af26-0c4a575523be","Type":"ContainerStarted","Data":"0de3dda5865d22dfaaa64e5fa999a3ad9fbb2614096c331c053ef905dfc54e5a"} Jul 10 00:17:32.649454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:32.649397 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r9kxr" podStartSLOduration=2.721162118 podStartE2EDuration="23.649378792s" totalImagesPullingTime="20.928216674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:32.648855582 +0000 UTC m=+24.675334286" watchObservedRunningTime="2026-07-10 00:17:32.649378792 +0000 UTC m=+24.675857496" Jul 10 00:17:34.507461 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.507372 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:34.507461 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.507415 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:34.507461 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.507433 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:34.508161 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:34.507605 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:34.508161 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:34.507709 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:34.508161 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:34.507759 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:34.639834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.639792 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" event={"ID":"04129ab2-4456-4acb-839a-b296be7fa6f5","Type":"ContainerStarted","Data":"670c1433d2c1dad38dbe4f5a941ce80a6d1612e2cb46f0fd095bb0cdd2525fb1"} Jul 10 00:17:34.646762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.646736 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:17:34.647170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.647148 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"689dad4898e33a8694121448115e606c2accc356cc0e3fc699d37d365e2cf09a"} Jul 10 00:17:34.658157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:34.658109 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tsx2d" podStartSLOduration=3.378661137 podStartE2EDuration="25.658096458s" totalImagesPullingTime="22.279435321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:34.657982248 +0000 UTC m=+26.684460953" watchObservedRunningTime="2026-07-10 00:17:34.658096458 +0000 UTC m=+26.684575162" Jul 10 00:17:36.507244 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:36.507200 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:36.507691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:36.507208 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:36.507691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:36.507338 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:36.507691 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:36.507330 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:36.507691 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:36.507419 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:36.507691 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:36.507525 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:37.655976 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.655730 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="bd9637bd0054cf30932ddaba47b344ef2765e8550c81d6fc3a633e8e783d94fd" exitCode=0 Jul 10 00:17:37.655976 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.655803 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"bd9637bd0054cf30932ddaba47b344ef2765e8550c81d6fc3a633e8e783d94fd"} Jul 10 00:17:37.659313 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.659299 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:17:37.659641 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.659623 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"c104c92e2c984e504c78bf09e964399286d9871bbbf7f0e1b6ea64aa97e2a36d"} Jul 10 00:17:37.659930 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.659912 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:37.660030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.660001 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:37.660091 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.660066 2572 scope.go:117] "RemoveContainer" containerID="a432cbe82c9161a3310e582109587fe455a31845dba6c308cb11a056cbf4e5c3" Jul 10 00:17:37.677579 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:37.677554 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:38.338622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.338587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:38.338872 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.338730 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 00:17:38.339288 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.339264 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hbxb" Jul 10 00:17:38.508776 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.508743 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:38.508977 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:38.508847 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:38.509227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.509210 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:38.509326 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:38.509310 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:38.509382 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.509356 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:38.509432 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:38.509419 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:38.665738 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.665669 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:17:38.666363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.666328 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" event={"ID":"5f623af4-76b4-464c-96ef-5e17791307a8","Type":"ContainerStarted","Data":"7af8e92a9afaafe64fffe4927287398698735f4e8158a0f50f2bb1ddb98fed1f"} Jul 10 00:17:38.666675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.666655 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:38.692079 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.692051 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:17:38.696372 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:38.696332 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" podStartSLOduration=8.694776048 podStartE2EDuration="29.696319447s" totalImagesPullingTime="21.001543399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:38.695007977 +0000 UTC m=+30.721486684" watchObservedRunningTime="2026-07-10 00:17:38.696319447 +0000 UTC m=+30.722798150" Jul 10 00:17:39.250638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.250609 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ql7b9"] Jul 10 00:17:39.250793 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.250720 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:39.250850 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:39.250812 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:39.253268 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.253246 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2c8lv"] Jul 10 00:17:39.253379 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.253335 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:39.253455 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:39.253436 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:39.253759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.253738 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n7czp"] Jul 10 00:17:39.253881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.253828 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:39.253944 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:39.253927 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:39.670534 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.670457 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="54ed0606ddec2b3edd7bfe0287fede2b4449de1a5620b968ea15ce34e30f39b1" exitCode=0 Jul 10 00:17:39.670863 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:39.670533 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"54ed0606ddec2b3edd7bfe0287fede2b4449de1a5620b968ea15ce34e30f39b1"} Jul 10 00:17:40.508124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:40.508096 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:40.508259 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:40.508233 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:41.507239 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:41.507088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:41.507647 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:41.507140 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:41.507647 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:41.507323 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:41.507647 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:41.507424 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:41.677174 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:41.677133 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="88c29bdf0c43d62077ebbc7f00b6643b870c3acefddf9452e57ed93cd617d01e" exitCode=0 Jul 10 00:17:41.677174 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:41.677176 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"88c29bdf0c43d62077ebbc7f00b6643b870c3acefddf9452e57ed93cd617d01e"} Jul 10 00:17:42.170814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:42.170783 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:42.171003 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:42.170839 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:42.171003 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.170930 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:42.171003 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.170930 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:42.171113 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.171008 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret podName:c2168193-a602-41e7-adc0-7c0a76072865 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:14.170989196 +0000 UTC m=+66.197467878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret") pod "global-pull-secret-syncer-2c8lv" (UID: "c2168193-a602-41e7-adc0-7c0a76072865") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:17:42.171113 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.171021 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:18:14.17101508 +0000 UTC m=+66.197493762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:17:42.372117 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:42.372082 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:42.372275 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.372243 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:17:42.372275 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.372260 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:17:42.372275 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.372274 2572 projected.go:196] Error preparing data for projected volume kube-api-access-lsrv8 for pod openshift-network-diagnostics/network-check-target-ql7b9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:42.372415 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.372336 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8 podName:8f3ca513-e532-4416-b4a8-31435e3d7d5c nodeName:}" failed. No retries permitted until 2026-07-10 00:18:14.372319366 +0000 UTC m=+66.398798051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsrv8" (UniqueName: "kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8") pod "network-check-target-ql7b9" (UID: "8f3ca513-e532-4416-b4a8-31435e3d7d5c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:17:42.507753 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:42.507717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:42.508283 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:42.507857 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:43.506992 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:43.506967 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:43.507151 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:43.507094 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:17:43.507151 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:43.507128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:43.507282 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:43.507202 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ql7b9" podUID="8f3ca513-e532-4416-b4a8-31435e3d7d5c" Jul 10 00:17:44.506999 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:44.506970 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:44.507455 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:44.507088 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2c8lv" podUID="c2168193-a602-41e7-adc0-7c0a76072865" Jul 10 00:17:45.198017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.197989 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-23.ec2.internal" event="NodeReady" Jul 10 00:17:45.198132 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.198112 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 00:17:45.229187 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.229156 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q4kw4"] Jul 10 00:17:45.231023 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.231004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.233438 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.233412 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-59kgw\"" Jul 10 00:17:45.233438 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.233423 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 00:17:45.233616 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.233442 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 00:17:45.244180 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.242575 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:17:45.244694 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.244669 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.247655 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247119 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q4kw4"] Jul 10 00:17:45.247655 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247476 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 00:17:45.247807 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247688 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 00:17:45.247807 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247762 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-npqnr"] Jul 10 00:17:45.247909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247815 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tf97q\"" Jul 10 00:17:45.247909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.247896 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 00:17:45.250622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.250604 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4xjzv"] Jul 10 00:17:45.250914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.250717 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.252859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.252843 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.253173 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.253154 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 00:17:45.253386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.253368 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 00:17:45.253470 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.253370 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 00:17:45.253470 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.253372 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ffdj8\"" Jul 10 00:17:45.253887 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.253867 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 00:17:45.255211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.255190 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p75jv\"" Jul 10 00:17:45.255500 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.255482 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 00:17:45.256070 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.256049 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 00:17:45.257153 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.257135 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:17:45.259496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.259479 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-npqnr"] Jul 10 00:17:45.263316 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.263298 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xjzv"] Jul 10 00:17:45.393709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f421c76d-3cf5-4d11-bd6c-95ae39675933-config-volume\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.393709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f421c76d-3cf5-4d11-bd6c-95ae39675933-tmp-dir\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.393709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393674 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wsb\" (UniqueName: \"kubernetes.io/projected/65c91095-98b9-46b3-8a68-62eeccdab145-kube-api-access-q7wsb\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.393709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccxr6\" (UniqueName: \"kubernetes.io/projected/f421c76d-3cf5-4d11-bd6c-95ae39675933-kube-api-access-ccxr6\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393795 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393835 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393865 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393912 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c91095-98b9-46b3-8a68-62eeccdab145-cert\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.394014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.393935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smjj9\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.394016 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe453deb-c767-4f25-8637-46a098957006-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.394294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.394046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.394294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.394076 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.394113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.394294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.394165 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c91095-98b9-46b3-8a68-62eeccdab145-cert\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.495346 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smjj9\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495348 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe453deb-c767-4f25-8637-46a098957006-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.495418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.495519 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.495499 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:17:45.495567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495520 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495548 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495662 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.495569 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:45.995548688 +0000 UTC m=+38.022027375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:17:45.495662 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.495653 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:17:45.495766 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.495668 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:17:45.495766 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.495734 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:17:45.995712824 +0000 UTC m=+38.022191507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:17:45.495859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495761 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495851 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f421c76d-3cf5-4d11-bd6c-95ae39675933-config-volume\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.495982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495872 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.495982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495875 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f421c76d-3cf5-4d11-bd6c-95ae39675933-tmp-dir\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.495982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495910 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wsb\" (UniqueName: \"kubernetes.io/projected/65c91095-98b9-46b3-8a68-62eeccdab145-kube-api-access-q7wsb\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.495982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495936 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.495995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccxr6\" (UniqueName: \"kubernetes.io/projected/f421c76d-3cf5-4d11-bd6c-95ae39675933-kube-api-access-ccxr6\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496055 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496079 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.496170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496113 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe453deb-c767-4f25-8637-46a098957006-nginx-conf\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:45.496370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496176 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f421c76d-3cf5-4d11-bd6c-95ae39675933-tmp-dir\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.496370 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.496341 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:17:45.496429 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:45.496391 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:45.996377652 +0000 UTC m=+38.022856349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:17:45.496483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.496451 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f421c76d-3cf5-4d11-bd6c-95ae39675933-config-volume\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.497186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.497116 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.497787 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.497764 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.500078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.500057 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.500176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.500075 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.500176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.500095 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65c91095-98b9-46b3-8a68-62eeccdab145-cert\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.505308 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.505281 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.505404 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.505315 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smjj9\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:45.505572 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.505552 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wsb\" (UniqueName: \"kubernetes.io/projected/65c91095-98b9-46b3-8a68-62eeccdab145-kube-api-access-q7wsb\") pod \"ingress-canary-npqnr\" (UID: \"65c91095-98b9-46b3-8a68-62eeccdab145\") " pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:45.506069 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.506046 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccxr6\" (UniqueName: \"kubernetes.io/projected/f421c76d-3cf5-4d11-bd6c-95ae39675933-kube-api-access-ccxr6\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:45.507760 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.507739 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:17:45.508343 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.508095 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:17:45.510178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.510162 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 00:17:45.510540 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.510522 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7p4r9\"" Jul 10 00:17:45.510626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.510589 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 00:17:45.510679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.510667 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7wp6\"" Jul 10 00:17:45.510783 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.510769 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 00:17:45.568114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:45.568084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-npqnr" Jul 10 00:17:46.001320 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:46.001273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:46.001364 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:46.001391 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001394 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001446 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001492 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:47.00147126 +0000 UTC m=+39.027949948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001509 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001516 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:47.001499141 +0000 UTC m=+39.027977838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:17:46.001522 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001525 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:17:46.001817 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:46.001575 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:17:47.001562938 +0000 UTC m=+39.028041620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:17:46.507302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:46.507270 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:17:46.510188 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:46.509983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 00:17:47.009220 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.009180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:47.009348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.009242 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:47.009348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.009259 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:47.009348 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009327 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:17:47.009446 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009356 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:17:47.009446 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009366 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:17:47.009446 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009389 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:49.009374933 +0000 UTC m=+41.035853616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:17:47.009446 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009390 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:17:47.009446 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009405 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:17:49.009394433 +0000 UTC m=+41.035873115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:17:47.009588 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:47.009452 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:49.009435615 +0000 UTC m=+41.035914299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:17:47.183145 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.182985 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-npqnr"] Jul 10 00:17:47.265543 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:17:47.265433 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c91095_98b9_46b3_8a68_62eeccdab145.slice/crio-889861a422052e4cc80608bb3e797315c77cd7bb6a61060a12b9e1ed9673c7b2 WatchSource:0}: Error finding container 889861a422052e4cc80608bb3e797315c77cd7bb6a61060a12b9e1ed9673c7b2: Status 404 returned error can't find the container with id 889861a422052e4cc80608bb3e797315c77cd7bb6a61060a12b9e1ed9673c7b2 Jul 10 00:17:47.696093 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.696021 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="e9e131d55df44a2ecd71f9716ff927ff35e3c6e2aa12089d885784033be63d61" exitCode=0 Jul 10 00:17:47.696491 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.696086 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"e9e131d55df44a2ecd71f9716ff927ff35e3c6e2aa12089d885784033be63d61"} Jul 10 00:17:47.697149 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.697128 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-npqnr" event={"ID":"65c91095-98b9-46b3-8a68-62eeccdab145","Type":"ContainerStarted","Data":"889861a422052e4cc80608bb3e797315c77cd7bb6a61060a12b9e1ed9673c7b2"} Jul 10 00:17:47.754528 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.754475 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:47.868655 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.868629 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:47.939416 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.939382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:47.998728 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:47.998619 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:48.677300 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:48.677261 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:48.702880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:48.702851 2572 generic.go:356] "Generic (PLEG): container finished" podID="45825668-2a70-4327-a7f5-6ad932ab0eb6" containerID="d2609c11398a987a6de91ec94e4a514b8f83b674ee22ab10ed946aaa729c1e09" exitCode=0 Jul 10 00:17:48.703271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:48.702884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerDied","Data":"d2609c11398a987a6de91ec94e4a514b8f83b674ee22ab10ed946aaa729c1e09"} Jul 10 00:17:48.777734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:48.777705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:49.025133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.025101 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:49.025291 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.025155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:49.025291 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.025172 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:49.025291 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025273 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:17:49.025401 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025349 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:53.025333688 +0000 UTC m=+45.051812370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:17:49.025401 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025274 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:17:49.025401 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025379 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:17:49.025401 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025273 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:17:49.025518 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025431 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:17:53.025412633 +0000 UTC m=+45.051891315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:17:49.025518 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:49.025443 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:17:53.025437857 +0000 UTC m=+45.051916539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:17:49.077884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.077845 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:49.126855 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.126824 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:49.709535 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.709288 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" event={"ID":"45825668-2a70-4327-a7f5-6ad932ab0eb6","Type":"ContainerStarted","Data":"71d8592d4e31f0f5f3166eec171d741a1c543a114459983ef061085b987be504"} Jul 10 00:17:49.732633 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.732582 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wb9hb" podStartSLOduration=11.06263919 podStartE2EDuration="40.732563209s" totalImagesPullingTime="29.669924019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:49.731367209 +0000 UTC m=+41.757845946" watchObservedRunningTime="2026-07-10 00:17:49.732563209 +0000 UTC m=+41.759041915" Jul 10 00:17:49.886615 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.886582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:49.937888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:49.937859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:50.275112 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:50.275072 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:50.324489 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:50.324459 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:50.713028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:50.712990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-npqnr" event={"ID":"65c91095-98b9-46b3-8a68-62eeccdab145","Type":"ContainerStarted","Data":"66b37cd5e8837c0db30388ca35c1d386bb789451a3678c693d54b8b00aeae0eb"} Jul 10 00:17:50.727967 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:50.727914 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-npqnr" podStartSLOduration=2.778633757 podStartE2EDuration="5.727900351s" totalImagesPullingTime="2.949266594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:50.727407705 +0000 UTC m=+42.753886410" watchObservedRunningTime="2026-07-10 00:17:50.727900351 +0000 UTC m=+42.754379055" Jul 10 00:17:51.077559 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:51.077521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:51.178108 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:51.178079 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:51.476312 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:51.476230 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:51.528145 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:51.528116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:53.055137 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:53.055100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:53.055153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:53.055170 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055254 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055260 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055269 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055276 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055316 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:01.055300779 +0000 UTC m=+53.081779461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055330 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:18:01.055323796 +0000 UTC m=+53.081802478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:17:53.055537 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:17:53.055339 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:01.055334423 +0000 UTC m=+53.081813104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:17:56.495243 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:56.495210 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:56.595024 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:56.594994 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:56.746244 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:56.746157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:56.850661 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:56.850628 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:56.914208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:56.914169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:57.014868 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:57.014829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:57.116481 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:57.116444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:17:57.217085 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:17:57.217054 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:01.112564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:01.112524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:01.112586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:01.112612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112673 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112727 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112743 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:17.112727455 +0000 UTC m=+69.139206137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112756 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112769 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112773 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:17.11276083 +0000 UTC m=+69.139239513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:18:01.113035 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:01.112818 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:18:17.112803163 +0000 UTC m=+69.139281859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:18:09.943914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:09.943875 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:09.996720 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:09.996683 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:10.413483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.413446 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:10.484451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.484426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:10.683744 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.683675 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prt82" Jul 10 00:18:10.829009 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.828975 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:10.929906 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.929880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:10.979935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:10.979864 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:11.277381 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:11.277348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:11.330151 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:11.330106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:12.486414 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:12.486378 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:12.537794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:12.537763 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:13.801223 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:13.801189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:14.203709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.203620 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:18:14.203709 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.203693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:18:14.206083 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.206067 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 00:18:14.206211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.206194 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 00:18:14.213992 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:14.213972 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:18:14.214050 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:14.214037 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:19:18.214021439 +0000 UTC m=+130.240500120 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : secret "metrics-daemon-secret" not found Jul 10 00:18:14.217852 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.217835 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2168193-a602-41e7-adc0-7c0a76072865-original-pull-secret\") pod \"global-pull-secret-syncer-2c8lv\" (UID: \"c2168193-a602-41e7-adc0-7c0a76072865\") " pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:18:14.405006 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.404969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:18:14.407247 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.407227 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 00:18:14.411339 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.411318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:14.417262 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.417249 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2c8lv" Jul 10 00:18:14.418059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.418034 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 00:18:14.429284 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.429261 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrv8\" (UniqueName: \"kubernetes.io/projected/8f3ca513-e532-4416-b4a8-31435e3d7d5c-kube-api-access-lsrv8\") pod \"network-check-target-ql7b9\" (UID: \"8f3ca513-e532-4416-b4a8-31435e3d7d5c\") " pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:18:14.533197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.533151 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2c8lv"] Jul 10 00:18:14.536261 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:18:14.536240 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2168193_a602_41e7_adc0_7c0a76072865.slice/crio-5ac5bd471b6e549dd1bf60cef5ea1a1bc3ea5d101bb799272d093a86bd53461e WatchSource:0}: Error finding container 5ac5bd471b6e549dd1bf60cef5ea1a1bc3ea5d101bb799272d093a86bd53461e: Status 404 returned error can't find the container with id 5ac5bd471b6e549dd1bf60cef5ea1a1bc3ea5d101bb799272d093a86bd53461e Jul 10 00:18:14.627131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.627101 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7p4r9\"" Jul 10 00:18:14.636079 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.636058 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:18:14.751210 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.751184 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ql7b9"] Jul 10 00:18:14.754371 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:18:14.754340 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3ca513_e532_4416_b4a8_31435e3d7d5c.slice/crio-1c9e0ea33786018d997653760c64cdaa01e3b4223c4d4cba086c85250fbebbe7 WatchSource:0}: Error finding container 1c9e0ea33786018d997653760c64cdaa01e3b4223c4d4cba086c85250fbebbe7: Status 404 returned error can't find the container with id 1c9e0ea33786018d997653760c64cdaa01e3b4223c4d4cba086c85250fbebbe7 Jul 10 00:18:14.767985 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.767941 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2c8lv" event={"ID":"c2168193-a602-41e7-adc0-7c0a76072865","Type":"ContainerStarted","Data":"5ac5bd471b6e549dd1bf60cef5ea1a1bc3ea5d101bb799272d093a86bd53461e"} Jul 10 00:18:14.768922 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:14.768893 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ql7b9" event={"ID":"8f3ca513-e532-4416-b4a8-31435e3d7d5c","Type":"ContainerStarted","Data":"1c9e0ea33786018d997653760c64cdaa01e3b4223c4d4cba086c85250fbebbe7"} Jul 10 00:18:17.123567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:17.123531 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:17.123612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123681 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:17.123744 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123810 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:49.12379014 +0000 UTC m=+101.150268826 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123863 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123874 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123737 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123924 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:49.123912432 +0000 UTC m=+101.150391117 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:18:17.124057 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:17.123977 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:18:49.123943471 +0000 UTC m=+101.150422159 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:18:19.480287 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:19.480245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:19.537579 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:19.537538 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:19.585159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:19.585134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:20.785462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:20.785429 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2c8lv" event={"ID":"c2168193-a602-41e7-adc0-7c0a76072865","Type":"ContainerStarted","Data":"22ca9b8a69a7e2b760feb1a3d4f8f30bd3f7cc2ac62445ff39f7238a3681bd8a"} Jul 10 00:18:20.786829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:20.786804 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ql7b9" event={"ID":"8f3ca513-e532-4416-b4a8-31435e3d7d5c","Type":"ContainerStarted","Data":"fb45691fc8cfdf246c4f9dce93501d631f55cdf71ef8e0a16cbe197994ec62f5"} Jul 10 00:18:20.786942 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:20.786931 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:18:20.813537 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:20.813498 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2c8lv" podStartSLOduration=66.067515484 podStartE2EDuration="1m11.813483119s" totalImagesPullingTime="5.745967635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:18:20.800237999 +0000 UTC m=+72.826716704" watchObservedRunningTime="2026-07-10 00:18:20.813483119 +0000 UTC m=+72.839961820" Jul 10 00:18:20.813746 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:20.813719 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ql7b9" podStartSLOduration=66.285922586 podStartE2EDuration="1m11.813710926s" totalImagesPullingTime="5.52778834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:18:20.813279165 +0000 UTC m=+72.839757893" watchObservedRunningTime="2026-07-10 00:18:20.813710926 +0000 UTC m=+72.840189630" Jul 10 00:18:23.030853 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:23.030816 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:23.130319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:23.130286 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:27.297247 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:27.297211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:27.345910 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:27.345877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:31.460599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:31.460481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:31.519815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:31.519783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:31.960228 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:31.960192 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:32.012180 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:32.012150 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:35.598466 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:35.598434 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:35.654217 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:35.654182 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:36.229332 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:36.229286 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:49.162190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:49.162146 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:49.162201 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162283 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162296 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:49.162321 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162344 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:53.162331296 +0000 UTC m=+165.188809978 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162370 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:53.162352721 +0000 UTC m=+165.188831404 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162389 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162399 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:18:49.162626 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:18:49.162445 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:19:53.162434435 +0000 UTC m=+165.188913118 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:18:49.936997 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:49.936963 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:49.988449 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:49.988414 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:51.791667 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:51.791639 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ql7b9" Jul 10 00:18:56.541712 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:56.541677 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:56.590791 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:56.590754 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:18:56.646763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:18:56.646740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:19:18.256295 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:18.256241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:19:18.256709 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:18.256375 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:19:18.256709 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:18.256445 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:21:20.256430736 +0000 UTC m=+252.282909418 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : secret "metrics-daemon-secret" not found Jul 10 00:19:21.962301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:21.962256 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:19:48.244257 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:48.244205 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" podUID="fe453deb-c767-4f25-8637-46a098957006" Jul 10 00:19:48.258422 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:48.258378 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-864b84b995-9x97q" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" Jul 10 00:19:48.273201 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:48.273166 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4xjzv" podUID="f421c76d-3cf5-4d11-bd6c-95ae39675933" Jul 10 00:19:48.530026 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:48.529989 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:19:48.988088 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:48.988007 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:19:48.988227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:48.988007 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:19:48.988227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:48.988013 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:19:53.196793 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:53.196754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:53.196805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.196885 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.196887 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:19:53.196921 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.196938 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls podName:f421c76d-3cf5-4d11-bd6c-95ae39675933 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:55.196925606 +0000 UTC m=+287.223404288 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls") pod "dns-default-4xjzv" (UID: "f421c76d-3cf5-4d11-bd6c-95ae39675933") : secret "dns-default-metrics-tls" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.196977 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert podName:fe453deb-c767-4f25-8637-46a098957006 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:55.196965648 +0000 UTC m=+287.223444341 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-q4kw4" (UID: "fe453deb-c767-4f25-8637-46a098957006") : secret "networking-console-plugin-cert" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.197014 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.197025 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864b84b995-9x97q: secret "image-registry-tls" not found Jul 10 00:19:53.197219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:19:53.197073 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls podName:bbf91f49-aaae-4790-855d-ec048f0a415d nodeName:}" failed. No retries permitted until 2026-07-10 00:21:55.197061003 +0000 UTC m=+287.223539690 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls") pod "image-registry-864b84b995-9x97q" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d") : secret "image-registry-tls" not found Jul 10 00:20:00.507339 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:20:00.507251 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:21:02.309586 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:02.309550 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:20.309817 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:20.309776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:21:20.310245 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:20.309920 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:21:20.310245 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:20.309998 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs podName:13e20826-95ee-455c-8405-f23540349d1e nodeName:}" failed. No retries permitted until 2026-07-10 00:23:22.309982242 +0000 UTC m=+374.336460924 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs") pod "network-metrics-daemon-n7czp" (UID: "13e20826-95ee-455c-8405-f23540349d1e") : secret "metrics-daemon-secret" not found Jul 10 00:21:28.062512 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.062477 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2"] Jul 10 00:21:28.065233 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.065217 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244"] Jul 10 00:21:28.065386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.065368 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.067821 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.067803 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-678f64b94d-llqjf"] Jul 10 00:21:28.067931 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.067915 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.068184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.068124 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 00:21:28.068184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.068125 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 00:21:28.068315 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.068142 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cjdtg\"" Jul 10 00:21:28.068368 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.068353 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 00:21:28.071488 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.071370 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:21:28.071579 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.071491 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 00:21:28.071701 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.071680 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 00:21:28.071767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.071746 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 00:21:28.071767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.071761 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 00:21:28.072106 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.072085 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-68ltp\"" Jul 10 00:21:28.072200 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.072161 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.076225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076190 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 00:21:28.076344 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076234 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 00:21:28.076344 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076277 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 00:21:28.076344 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076234 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 00:21:28.076555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076428 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 00:21:28.076555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076466 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 00:21:28.076555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.076510 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b4n4c\"" Jul 10 00:21:28.077497 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.077449 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244"] Jul 10 00:21:28.078523 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.078500 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2"] Jul 10 00:21:28.082161 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.082142 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-678f64b94d-llqjf"] Jul 10 00:21:28.166270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166238 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.166270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166270 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/879948e0-3f63-4ada-b677-1c55541823d3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.166496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-default-certificate\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.166496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166308 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.166496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166386 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e604290b-7f97-494d-82ad-55d29971c045-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.166496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e604290b-7f97-494d-82ad-55d29971c045-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.166496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxj6b\" (UniqueName: \"kubernetes.io/projected/fdc51d59-fb65-4059-a734-ceeb6b050aec-kube-api-access-zxj6b\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.166679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5n8s\" (UniqueName: \"kubernetes.io/projected/879948e0-3f63-4ada-b677-1c55541823d3-kube-api-access-k5n8s\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.166679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166538 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hfch\" (UniqueName: \"kubernetes.io/projected/e604290b-7f97-494d-82ad-55d29971c045-kube-api-access-2hfch\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.166679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166587 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-stats-auth\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.166679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.166619 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.175716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.175693 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ppbl"] Jul 10 00:21:28.178401 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.178388 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7"] Jul 10 00:21:28.178522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.178507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.181013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.180995 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.181551 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.181536 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 00:21:28.181626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.181577 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 00:21:28.181922 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.181905 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 00:21:28.184599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.184577 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 00:21:28.184599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.184588 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 00:21:28.184770 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.184662 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-j8s4w\"" Jul 10 00:21:28.184770 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.184683 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 00:21:28.184857 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.184775 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:21:28.186423 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.186402 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-794mn\"" Jul 10 00:21:28.186914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.186890 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 00:21:28.194054 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.194035 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7"] Jul 10 00:21:28.194993 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.194976 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 00:21:28.198758 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.198732 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ppbl"] Jul 10 00:21:28.267211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267180 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.267211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267210 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95732d1-a141-48f0-afff-e90eefae17d3-serving-cert\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267240 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hfch\" (UniqueName: \"kubernetes.io/projected/e604290b-7f97-494d-82ad-55d29971c045-kube-api-access-2hfch\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmrx\" (UniqueName: \"kubernetes.io/projected/d95732d1-a141-48f0-afff-e90eefae17d3-kube-api-access-4mmrx\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267289 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267304 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-snapshots\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-stats-auth\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.267453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267443 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/879948e0-3f63-4ada-b677-1c55541823d3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-default-certificate\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.267511 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267563 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e604290b-7f97-494d-82ad-55d29971c045-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.267592 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:28.767571197 +0000 UTC m=+260.794049879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : secret "router-metrics-certs-default" not found Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.267608 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267626 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e604290b-7f97-494d-82ad-55d29971c045-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.267677 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:28.767658814 +0000 UTC m=+260.794137498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxj6b\" (UniqueName: \"kubernetes.io/projected/fdc51d59-fb65-4059-a734-ceeb6b050aec-kube-api-access-zxj6b\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.267727 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:28.767713839 +0000 UTC m=+260.794192523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267750 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97k72\" (UniqueName: \"kubernetes.io/projected/de72b07c-46c2-450d-aa9c-3c0b983e5beb-kube-api-access-97k72\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.267796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267781 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de72b07c-46c2-450d-aa9c-3c0b983e5beb-serving-cert\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.268363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5n8s\" (UniqueName: \"kubernetes.io/projected/879948e0-3f63-4ada-b677-1c55541823d3-kube-api-access-k5n8s\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.268363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267839 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-tmp\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.268363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.267891 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de72b07c-46c2-450d-aa9c-3c0b983e5beb-config\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.268363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.268309 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e604290b-7f97-494d-82ad-55d29971c045-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.268513 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.268428 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/879948e0-3f63-4ada-b677-1c55541823d3-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.269980 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.269942 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-stats-auth\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.270071 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.270018 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e604290b-7f97-494d-82ad-55d29971c045-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.270132 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.270116 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-default-certificate\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.275631 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.275613 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxj6b\" (UniqueName: \"kubernetes.io/projected/fdc51d59-fb65-4059-a734-ceeb6b050aec-kube-api-access-zxj6b\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.278635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.278606 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5n8s\" (UniqueName: \"kubernetes.io/projected/879948e0-3f63-4ada-b677-1c55541823d3-kube-api-access-k5n8s\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.278746 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.278730 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hfch\" (UniqueName: \"kubernetes.io/projected/e604290b-7f97-494d-82ad-55d29971c045-kube-api-access-2hfch\") pod \"kube-storage-version-migrator-operator-799cb7f958-kz244\" (UID: \"e604290b-7f97-494d-82ad-55d29971c045\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.318369 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.318300 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:28.368587 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368558 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97k72\" (UniqueName: \"kubernetes.io/projected/de72b07c-46c2-450d-aa9c-3c0b983e5beb-kube-api-access-97k72\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.368705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de72b07c-46c2-450d-aa9c-3c0b983e5beb-serving-cert\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.368705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-tmp\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.368705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de72b07c-46c2-450d-aa9c-3c0b983e5beb-config\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.368705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368675 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.368705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368696 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95732d1-a141-48f0-afff-e90eefae17d3-serving-cert\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.368942 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368721 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmrx\" (UniqueName: \"kubernetes.io/projected/d95732d1-a141-48f0-afff-e90eefae17d3-kube-api-access-4mmrx\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.369025 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.368991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.369087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.369029 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-snapshots\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.369148 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.369097 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-tmp\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.369353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.369330 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/de72b07c-46c2-450d-aa9c-3c0b983e5beb-config\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.369522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.369505 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.369900 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.369878 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d95732d1-a141-48f0-afff-e90eefae17d3-snapshots\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.370567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.370543 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d95732d1-a141-48f0-afff-e90eefae17d3-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.371029 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.371011 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de72b07c-46c2-450d-aa9c-3c0b983e5beb-serving-cert\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.371356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.371341 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d95732d1-a141-48f0-afff-e90eefae17d3-serving-cert\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.375744 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.375724 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97k72\" (UniqueName: \"kubernetes.io/projected/de72b07c-46c2-450d-aa9c-3c0b983e5beb-kube-api-access-97k72\") pod \"service-ca-operator-7695fc6c8c-45qm7\" (UID: \"de72b07c-46c2-450d-aa9c-3c0b983e5beb\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.376072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.376049 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmrx\" (UniqueName: \"kubernetes.io/projected/d95732d1-a141-48f0-afff-e90eefae17d3-kube-api-access-4mmrx\") pod \"insights-operator-68d9dd94cf-7ppbl\" (UID: \"d95732d1-a141-48f0-afff-e90eefae17d3\") " pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.384769 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.384754 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" Jul 10 00:21:28.418613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.418579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:28.488451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.488425 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" Jul 10 00:21:28.493574 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.493550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" Jul 10 00:21:28.505593 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.505566 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244"] Jul 10 00:21:28.509084 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:28.509056 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode604290b_7f97_494d_82ad_55d29971c045.slice/crio-0a78e8d22670068445f7a29207aad103dd133f4829e8a4e78fb3b50df484be19 WatchSource:0}: Error finding container 0a78e8d22670068445f7a29207aad103dd133f4829e8a4e78fb3b50df484be19: Status 404 returned error can't find the container with id 0a78e8d22670068445f7a29207aad103dd133f4829e8a4e78fb3b50df484be19 Jul 10 00:21:28.624803 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.624781 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7ppbl"] Jul 10 00:21:28.627488 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:28.627460 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95732d1_a141_48f0_afff_e90eefae17d3.slice/crio-1c7f5ca528cfd4f7d3a18d011e88bce9a35c2591efcd6a61792eba7b36a0338f WatchSource:0}: Error finding container 1c7f5ca528cfd4f7d3a18d011e88bce9a35c2591efcd6a61792eba7b36a0338f: Status 404 returned error can't find the container with id 1c7f5ca528cfd4f7d3a18d011e88bce9a35c2591efcd6a61792eba7b36a0338f Jul 10 00:21:28.638669 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.638647 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7"] Jul 10 00:21:28.641091 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:28.641069 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde72b07c_46c2_450d_aa9c_3c0b983e5beb.slice/crio-f4f7f66b3fc4d55ebcd6b3e2506de4d7bcce683365daf048151af9a48f4a3506 WatchSource:0}: Error finding container f4f7f66b3fc4d55ebcd6b3e2506de4d7bcce683365daf048151af9a48f4a3506: Status 404 returned error can't find the container with id f4f7f66b3fc4d55ebcd6b3e2506de4d7bcce683365daf048151af9a48f4a3506 Jul 10 00:21:28.773496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.773460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.773643 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.773510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:28.773643 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:28.773548 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:28.773643 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.773610 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 00:21:28.773740 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.773676 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:29.773658662 +0000 UTC m=+261.800137347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : secret "router-metrics-certs-default" not found Jul 10 00:21:28.773740 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.773684 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:28.773740 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.773693 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:29.773683045 +0000 UTC m=+261.800161727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:28.773740 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:28.773736 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:29.773720558 +0000 UTC m=+261.800199240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:29.222247 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.222206 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" event={"ID":"e604290b-7f97-494d-82ad-55d29971c045","Type":"ContainerStarted","Data":"0a78e8d22670068445f7a29207aad103dd133f4829e8a4e78fb3b50df484be19"} Jul 10 00:21:29.223567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.223511 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" event={"ID":"de72b07c-46c2-450d-aa9c-3c0b983e5beb","Type":"ContainerStarted","Data":"f4f7f66b3fc4d55ebcd6b3e2506de4d7bcce683365daf048151af9a48f4a3506"} Jul 10 00:21:29.225073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.225045 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" event={"ID":"d95732d1-a141-48f0-afff-e90eefae17d3","Type":"ContainerStarted","Data":"1c7f5ca528cfd4f7d3a18d011e88bce9a35c2591efcd6a61792eba7b36a0338f"} Jul 10 00:21:29.783473 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.783381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:29.783473 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.783449 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:29.783725 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:29.783534 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:29.783725 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.783572 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:29.783725 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:29.783603 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:31.783587254 +0000 UTC m=+263.810065936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:29.783725 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:29.783676 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 00:21:29.783725 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:29.783726 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:31.783709266 +0000 UTC m=+263.810187964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : secret "router-metrics-certs-default" not found Jul 10 00:21:29.784023 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:29.783802 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:31.783792174 +0000 UTC m=+263.810270857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:29.985720 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:29.985659 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:30.037078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:30.036984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:31.800286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:31.800249 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:31.800317 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:31.800351 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:31.800415 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:31.800471 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:31.800491 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:35.800474156 +0000 UTC m=+267.826952841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:31.800512 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:35.800502593 +0000 UTC m=+267.826981278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : secret "router-metrics-certs-default" not found Jul 10 00:21:31.800679 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:31.800528 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:35.800519732 +0000 UTC m=+267.826998418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:32.057899 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.057812 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.113269 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.113235 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.188587 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.188554 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k9z4k"] Jul 10 00:21:32.192538 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.192512 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.197731 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.197710 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 00:21:32.197866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.197778 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 00:21:32.197866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.197846 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6fk5l\"" Jul 10 00:21:32.208803 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.208774 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9z4k"] Jul 10 00:21:32.233825 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.233792 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.235838 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.235818 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" event={"ID":"e604290b-7f97-494d-82ad-55d29971c045","Type":"ContainerStarted","Data":"b4216506689f3c98e268d5abfb8e313f6cdc82ba15eb735d73d64113107c8bb8"} Jul 10 00:21:32.237688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.237670 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" event={"ID":"de72b07c-46c2-450d-aa9c-3c0b983e5beb","Type":"ContainerStarted","Data":"18a13e849eb7670b26356162b0c81c04759b8ffe1b7c98e1fadca656bcd5c9e9"} Jul 10 00:21:32.239238 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.239206 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" event={"ID":"d95732d1-a141-48f0-afff-e90eefae17d3","Type":"ContainerStarted","Data":"a6f9d5bd8af7724fcd9fb40f3864a18151eb034e564f3e2862219b65449713b2"} Jul 10 00:21:32.252624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.252580 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" podStartSLOduration=1.457668004 podStartE2EDuration="4.252563866s" totalImagesPullingTime="2.794895862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:32.251623547 +0000 UTC m=+264.278102252" watchObservedRunningTime="2026-07-10 00:21:32.252563866 +0000 UTC m=+264.279042572" Jul 10 00:21:32.268466 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.268419 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" podStartSLOduration=1.58511727 podStartE2EDuration="4.268404851s" totalImagesPullingTime="2.683287581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:32.267343925 +0000 UTC m=+264.293822637" watchObservedRunningTime="2026-07-10 00:21:32.268404851 +0000 UTC m=+264.294883554" Jul 10 00:21:32.281178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.281134 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" podStartSLOduration=1.613225952 podStartE2EDuration="4.281109654s" totalImagesPullingTime="2.667883702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:32.280810831 +0000 UTC m=+264.307289538" watchObservedRunningTime="2026-07-10 00:21:32.281109654 +0000 UTC m=+264.307588357" Jul 10 00:21:32.299059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.299026 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.304145 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.304115 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.304271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.304201 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jcmc\" (UniqueName: \"kubernetes.io/projected/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-api-access-8jcmc\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.304330 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.304295 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff07e5ee-efae-401a-8f45-d9546536e6a5-crio-socket\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.304410 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.304395 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff07e5ee-efae-401a-8f45-d9546536e6a5-data-volume\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.304497 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.304480 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.404931 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.405053 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jcmc\" (UniqueName: \"kubernetes.io/projected/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-api-access-8jcmc\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.405129 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:32.905101526 +0000 UTC m=+264.931580224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : configmap "kube-rbac-proxy" not found Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.405177 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff07e5ee-efae-401a-8f45-d9546536e6a5-crio-socket\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.405240 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:32.905221467 +0000 UTC m=+264.931700163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:21:32.405290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405271 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff07e5ee-efae-401a-8f45-d9546536e6a5-crio-socket\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405305 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff07e5ee-efae-401a-8f45-d9546536e6a5-data-volume\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.405741 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.405720 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff07e5ee-efae-401a-8f45-d9546536e6a5-data-volume\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.413083 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.413064 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jcmc\" (UniqueName: \"kubernetes.io/projected/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-api-access-8jcmc\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.497366 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.497333 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.571411 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.571374 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.625407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.625377 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.741550 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.741474 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.842197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.842166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:32.909628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.909593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.909789 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:32.909634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:32.909789 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.909703 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:21:32.909789 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.909730 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:21:32.909789 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.909762 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:33.909746846 +0000 UTC m=+265.936225529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : configmap "kube-rbac-proxy" not found Jul 10 00:21:32.909789 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:32.909776 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:33.909769438 +0000 UTC m=+265.936248121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:21:33.918567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:33.918518 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:33.918567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:33.918575 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:33.919098 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:33.918640 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:21:33.919098 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:33.918688 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:21:33.919098 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:33.918698 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:35.918684081 +0000 UTC m=+267.945162766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : configmap "kube-rbac-proxy" not found Jul 10 00:21:33.919098 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:33.918736 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:35.918724988 +0000 UTC m=+267.945203669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:21:34.381623 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:34.381597 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v8t9f_5fe2c04e-788c-49c7-bdb0-40d66299f6f9/dns-node-resolver/0.log" Jul 10 00:21:34.382963 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:34.382927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:34.981355 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:34.981328 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dmbpk_96c8aa3a-d36e-42fd-b36f-54a04b7d24d4/node-ca/0.log" Jul 10 00:21:35.259622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.259593 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:35.835250 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.835220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:35.835436 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.835262 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:35.835436 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.835344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:35.835436 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.835373 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:35.835436 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.835426 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 00:21:35.835579 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.835442 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:43.835427142 +0000 UTC m=+275.861905824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:35.835579 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.835459 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:43.835449321 +0000 UTC m=+275.861928003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : secret "router-metrics-certs-default" not found Jul 10 00:21:35.835579 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.835471 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:43.835465499 +0000 UTC m=+275.861944180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:35.935971 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.935925 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:35.936142 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.935993 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:35.936142 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.936047 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:21:35.936142 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.936114 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:39.936098892 +0000 UTC m=+271.962577590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : configmap "kube-rbac-proxy" not found Jul 10 00:21:35.936251 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.936138 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:21:35.936251 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:35.936191 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:39.93617785 +0000 UTC m=+271.962656531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:21:35.982031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:35.982002 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-npqnr_65c91095-98b9-46b3-8a68-62eeccdab145/serve-healthcheck-canary/0.log" Jul 10 00:21:37.785220 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:37.785188 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:37.893688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:37.893653 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:37.994799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:37.994741 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.113039 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.112939 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.214407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.214375 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.315994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.315945 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.450943 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.450867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.803017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.802943 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:38.994506 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:38.994473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:39.976641 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:39.976606 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:39.977055 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:39.976667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:39.977055 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:39.976734 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:21:39.977055 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:39.976789 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:21:39.977055 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:39.976810 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:47.976794145 +0000 UTC m=+280.003272827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : configmap "kube-rbac-proxy" not found Jul 10 00:21:39.977055 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:39.976842 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls podName:ff07e5ee-efae-401a-8f45-d9546536e6a5 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:47.976830244 +0000 UTC m=+280.003308926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k9z4k" (UID: "ff07e5ee-efae-401a-8f45-d9546536e6a5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:21:39.997391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:39.997361 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:40.393902 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:40.393863 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:40.603363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:40.603334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:40.805789 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:40.805758 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:41.193456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:41.193379 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:41.312547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:41.312515 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:41.790008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:41.789973 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:41.890829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:41.890800 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:41.942593 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:41.942561 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:42.198299 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:42.198189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:43.605811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.605771 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:43.792526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.792485 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:43.912061 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.911985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:43.912061 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.912035 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:43.912309 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:43.912128 2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:43.912309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.912162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:43.912309 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:43.912205 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls podName:879948e0-3f63-4ada-b677-1c55541823d3 nodeName:}" failed. No retries permitted until 2026-07-10 00:21:59.912186911 +0000 UTC m=+291.938665598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-d4nf2" (UID: "879948e0-3f63-4ada-b677-1c55541823d3") : secret "cluster-monitoring-operator-tls" not found Jul 10 00:21:43.912309 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:43.912224 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle podName:fdc51d59-fb65-4059-a734-ceeb6b050aec nodeName:}" failed. No retries permitted until 2026-07-10 00:21:59.912215042 +0000 UTC m=+291.938693727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle") pod "router-default-678f64b94d-llqjf" (UID: "fdc51d59-fb65-4059-a734-ceeb6b050aec") : configmap references non-existent config key: service-ca.crt Jul 10 00:21:43.914593 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:43.914571 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc51d59-fb65-4059-a734-ceeb6b050aec-metrics-certs\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:44.235655 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:44.235540 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:44.432802 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:44.432751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:44.594434 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:44.594407 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:44.996367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:44.996281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:45.198267 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:45.198234 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:45.463146 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:45.463114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:45.792859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:45.792825 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:46.414068 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:46.414033 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:46.604888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:46.604859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:46.995624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:46.995595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:47.197301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:47.197269 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:47.596047 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:47.596016 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:48.050106 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.050077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:48.050275 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.050119 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:48.050639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.050617 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff07e5ee-efae-401a-8f45-d9546536e6a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:48.052499 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.052481 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff07e5ee-efae-401a-8f45-d9546536e6a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k9z4k\" (UID: \"ff07e5ee-efae-401a-8f45-d9546536e6a5\") " pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:48.102815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.102787 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k9z4k" Jul 10 00:21:48.194314 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.194289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:48.224307 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.224283 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k9z4k"] Jul 10 00:21:48.226423 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:48.226391 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff07e5ee_efae_401a_8f45_d9546536e6a5.slice/crio-8db01fd5bf58a598e5c5e67ba69b69de63908de83c7fec7add6e2084d141acbe WatchSource:0}: Error finding container 8db01fd5bf58a598e5c5e67ba69b69de63908de83c7fec7add6e2084d141acbe: Status 404 returned error can't find the container with id 8db01fd5bf58a598e5c5e67ba69b69de63908de83c7fec7add6e2084d141acbe Jul 10 00:21:48.285870 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.285837 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9z4k" event={"ID":"ff07e5ee-efae-401a-8f45-d9546536e6a5","Type":"ContainerStarted","Data":"8db01fd5bf58a598e5c5e67ba69b69de63908de83c7fec7add6e2084d141acbe"} Jul 10 00:21:48.396652 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.396579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:48.594194 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:48.594155 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:51.988985 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:51.988925 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-864b84b995-9x97q" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" Jul 10 00:21:51.988985 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:51.988940 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4xjzv" podUID="f421c76d-3cf5-4d11-bd6c-95ae39675933" Jul 10 00:21:51.988985 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:21:51.988980 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" podUID="fe453deb-c767-4f25-8637-46a098957006" Jul 10 00:21:52.300281 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:52.300222 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:21:52.300281 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:52.300249 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9z4k" event={"ID":"ff07e5ee-efae-401a-8f45-d9546536e6a5","Type":"ContainerStarted","Data":"60404c0a679855fb18e12809d8e75f7e48283015946ac5906c6a09d965535adc"} Jul 10 00:21:52.300483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:52.300258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:21:52.300483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:52.300452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:21:53.305368 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:53.305334 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9z4k" event={"ID":"ff07e5ee-efae-401a-8f45-d9546536e6a5","Type":"ContainerStarted","Data":"a48b3d4f2ebd502106b1359f232bbd3301213a90f02fc76828fdcae74e39938f"} Jul 10 00:21:55.028799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.028761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:55.129392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.129359 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:55.214432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.214396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:21:55.214628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.214463 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:21:55.214628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.214504 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:21:55.217264 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.217238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f421c76d-3cf5-4d11-bd6c-95ae39675933-metrics-tls\") pod \"dns-default-4xjzv\" (UID: \"f421c76d-3cf5-4d11-bd6c-95ae39675933\") " pod="openshift-dns/dns-default-4xjzv" Jul 10 00:21:55.217406 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.217289 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"image-registry-864b84b995-9x97q\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:21:55.217406 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.217310 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fe453deb-c767-4f25-8637-46a098957006-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-q4kw4\" (UID: \"fe453deb-c767-4f25-8637-46a098957006\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:21:55.304221 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.304140 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-59kgw\"" Jul 10 00:21:55.304386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.304218 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tf97q\"" Jul 10 00:21:55.304386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.304308 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p75jv\"" Jul 10 00:21:55.311120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.311086 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:21:55.315329 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.312062 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" Jul 10 00:21:55.315329 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.312611 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:21:55.898619 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:55.898587 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:21:55.900293 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:55.900268 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf91f49_aaae_4790_855d_ec048f0a415d.slice/crio-7e08fde3145f657a29c57750a42d3b68c2300543a09827e1f09176f0a735af72 WatchSource:0}: Error finding container 7e08fde3145f657a29c57750a42d3b68c2300543a09827e1f09176f0a735af72: Status 404 returned error can't find the container with id 7e08fde3145f657a29c57750a42d3b68c2300543a09827e1f09176f0a735af72 Jul 10 00:21:56.118842 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.118763 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-q4kw4"] Jul 10 00:21:56.121925 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:56.121884 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe453deb_c767_4f25_8637_46a098957006.slice/crio-65dde096eaaacbd6252c151c5a1d2523127b0ef74618ae06885f6732b178b6ee WatchSource:0}: Error finding container 65dde096eaaacbd6252c151c5a1d2523127b0ef74618ae06885f6732b178b6ee: Status 404 returned error can't find the container with id 65dde096eaaacbd6252c151c5a1d2523127b0ef74618ae06885f6732b178b6ee Jul 10 00:21:56.122074 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.122057 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xjzv"] Jul 10 00:21:56.124526 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:21:56.124503 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf421c76d_3cf5_4d11_bd6c_95ae39675933.slice/crio-d9431672f929b8003c9fd5aee712d99168592fd80596b2bece577bca21b3b7c0 WatchSource:0}: Error finding container d9431672f929b8003c9fd5aee712d99168592fd80596b2bece577bca21b3b7c0: Status 404 returned error can't find the container with id d9431672f929b8003c9fd5aee712d99168592fd80596b2bece577bca21b3b7c0 Jul 10 00:21:56.321439 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.321401 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864b84b995-9x97q" event={"ID":"bbf91f49-aaae-4790-855d-ec048f0a415d","Type":"ContainerStarted","Data":"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540"} Jul 10 00:21:56.321604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.321458 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864b84b995-9x97q" event={"ID":"bbf91f49-aaae-4790-855d-ec048f0a415d","Type":"ContainerStarted","Data":"7e08fde3145f657a29c57750a42d3b68c2300543a09827e1f09176f0a735af72"} Jul 10 00:21:56.321604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.321506 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:21:56.322601 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.322552 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" event={"ID":"fe453deb-c767-4f25-8637-46a098957006","Type":"ContainerStarted","Data":"65dde096eaaacbd6252c151c5a1d2523127b0ef74618ae06885f6732b178b6ee"} Jul 10 00:21:56.323670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.323642 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xjzv" event={"ID":"f421c76d-3cf5-4d11-bd6c-95ae39675933","Type":"ContainerStarted","Data":"d9431672f929b8003c9fd5aee712d99168592fd80596b2bece577bca21b3b7c0"} Jul 10 00:21:56.325606 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.325582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k9z4k" event={"ID":"ff07e5ee-efae-401a-8f45-d9546536e6a5","Type":"ContainerStarted","Data":"62670edc7b93e2ea4622e9caf9b5201574ab968153aa6736745183776dfbd4c3"} Jul 10 00:21:56.348282 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.348238 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-864b84b995-9x97q" podStartSLOduration=273.348226564 podStartE2EDuration="4m33.348226564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:56.347212388 +0000 UTC m=+288.373691105" watchObservedRunningTime="2026-07-10 00:21:56.348226564 +0000 UTC m=+288.374705268" Jul 10 00:21:56.363782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:56.363743 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k9z4k" podStartSLOduration=16.902889608 podStartE2EDuration="24.363734375s" totalImagesPullingTime="7.460844767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:21:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:56.362121685 +0000 UTC m=+288.388600468" watchObservedRunningTime="2026-07-10 00:21:56.363734375 +0000 UTC m=+288.390213079" Jul 10 00:21:57.381472 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:57.381434 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:57.473783 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:57.473753 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:58.334157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.334061 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xjzv" event={"ID":"f421c76d-3cf5-4d11-bd6c-95ae39675933","Type":"ContainerStarted","Data":"efb522928d498a23a12c696199daaf5971d6017ee737beea936e811ba4fcff65"} Jul 10 00:21:58.334157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.334123 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xjzv" event={"ID":"f421c76d-3cf5-4d11-bd6c-95ae39675933","Type":"ContainerStarted","Data":"908af8ec2fc13f1ee8e018984fb2d58da7ca3ef2af1fd189ea509c82f39904db"} Jul 10 00:21:58.334157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.334154 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:21:58.335474 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.335447 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" event={"ID":"fe453deb-c767-4f25-8637-46a098957006","Type":"ContainerStarted","Data":"2c310cc184158c91d9734078e78059bc6172fdc48409e32e6a31eb10fec6b9e0"} Jul 10 00:21:58.349834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.349789 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4xjzv" podStartSLOduration=251.407202841 podStartE2EDuration="4m13.349774744s" totalImagesPullingTime="1.942571903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:58.348624792 +0000 UTC m=+290.375103518" watchObservedRunningTime="2026-07-10 00:21:58.349774744 +0000 UTC m=+290.376253425" Jul 10 00:21:58.362325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.362285 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-q4kw4" podStartSLOduration=265.986277233 podStartE2EDuration="4m27.36227072s" totalImagesPullingTime="1.375993487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:58.361640045 +0000 UTC m=+290.388118748" watchObservedRunningTime="2026-07-10 00:21:58.36227072 +0000 UTC m=+290.388749423" Jul 10 00:21:58.444460 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.444401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:58.498647 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.498613 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:58.554738 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:58.554705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:21:59.950496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:59.950455 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:21:59.950496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:59.950500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:59.951094 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:59.951077 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc51d59-fb65-4059-a734-ceeb6b050aec-service-ca-bundle\") pod \"router-default-678f64b94d-llqjf\" (UID: \"fdc51d59-fb65-4059-a734-ceeb6b050aec\") " pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:21:59.952884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:21:59.952860 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/879948e0-3f63-4ada-b677-1c55541823d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-d4nf2\" (UID: \"879948e0-3f63-4ada-b677-1c55541823d3\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:22:00.177897 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.177862 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" Jul 10 00:22:00.190076 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.190050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:22:00.308494 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.308374 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2"] Jul 10 00:22:00.311046 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:00.311022 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879948e0_3f63_4ada_b677_1c55541823d3.slice/crio-7129c12ce0875d784c265719560b15277f715ad4255b3f4c8f482fff4d020124 WatchSource:0}: Error finding container 7129c12ce0875d784c265719560b15277f715ad4255b3f4c8f482fff4d020124: Status 404 returned error can't find the container with id 7129c12ce0875d784c265719560b15277f715ad4255b3f4c8f482fff4d020124 Jul 10 00:22:00.327211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.327191 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-678f64b94d-llqjf"] Jul 10 00:22:00.329678 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:00.329659 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc51d59_fb65_4059_a734_ceeb6b050aec.slice/crio-1a23cf0737229e25ebfc9f83e9340f5ab43c9ed6cd1e6dc61b17c490074067c0 WatchSource:0}: Error finding container 1a23cf0737229e25ebfc9f83e9340f5ab43c9ed6cd1e6dc61b17c490074067c0: Status 404 returned error can't find the container with id 1a23cf0737229e25ebfc9f83e9340f5ab43c9ed6cd1e6dc61b17c490074067c0 Jul 10 00:22:00.342700 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.342672 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" event={"ID":"879948e0-3f63-4ada-b677-1c55541823d3","Type":"ContainerStarted","Data":"7129c12ce0875d784c265719560b15277f715ad4255b3f4c8f482fff4d020124"} Jul 10 00:22:00.343794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:00.343774 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-678f64b94d-llqjf" event={"ID":"fdc51d59-fb65-4059-a734-ceeb6b050aec","Type":"ContainerStarted","Data":"1a23cf0737229e25ebfc9f83e9340f5ab43c9ed6cd1e6dc61b17c490074067c0"} Jul 10 00:22:01.348817 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:01.348770 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-678f64b94d-llqjf" event={"ID":"fdc51d59-fb65-4059-a734-ceeb6b050aec","Type":"ContainerStarted","Data":"d62c4ca55f0d833b1c1e6e5efa45c7d8434a69debc6e4cdd74ba04b06cd2e762"} Jul 10 00:22:01.368138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:01.368092 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-678f64b94d-llqjf" podStartSLOduration=33.368072128 podStartE2EDuration="33.368072128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:01.366405184 +0000 UTC m=+293.392883911" watchObservedRunningTime="2026-07-10 00:22:01.368072128 +0000 UTC m=+293.394550833" Jul 10 00:22:01.509459 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:01.509425 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:01.567133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:01.567097 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:02.191011 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.190975 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:22:02.193608 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.193570 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:22:02.263920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.263892 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:02.354374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.354337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" event={"ID":"879948e0-3f63-4ada-b677-1c55541823d3","Type":"ContainerStarted","Data":"a430820b6bdf4c8c655e548b5d4aa10ba2c86499f7aa335d9ff3156fe4c57f5c"} Jul 10 00:22:02.354804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.354528 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:22:02.355736 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.355719 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-678f64b94d-llqjf" Jul 10 00:22:02.370462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.370420 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-d4nf2" podStartSLOduration=32.728740933 podStartE2EDuration="34.370406599s" totalImagesPullingTime="1.641665666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:02.369087257 +0000 UTC m=+294.395565980" watchObservedRunningTime="2026-07-10 00:22:02.370406599 +0000 UTC m=+294.396885302" Jul 10 00:22:02.434990 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.434933 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:02.437446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.437421 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg"] Jul 10 00:22:02.440447 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.440431 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:02.442517 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.442499 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7cttj\"" Jul 10 00:22:02.442698 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.442681 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 00:22:02.447025 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.447004 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg"] Jul 10 00:22:02.470891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.470864 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82768138-6201-43c1-901a-915be3e69f51-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mg6hg\" (UID: \"82768138-6201-43c1-901a-915be3e69f51\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:02.555616 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.555534 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:02.571659 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.571631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82768138-6201-43c1-901a-915be3e69f51-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mg6hg\" (UID: \"82768138-6201-43c1-901a-915be3e69f51\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:02.574154 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.574130 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82768138-6201-43c1-901a-915be3e69f51-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mg6hg\" (UID: \"82768138-6201-43c1-901a-915be3e69f51\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:02.752297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.752255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:02.873920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:02.873890 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg"] Jul 10 00:22:02.876535 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:02.876510 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82768138_6201_43c1_901a_915be3e69f51.slice/crio-5ce37a9da4132b2ae206771aa4a276c64f2ee25d52d825eaa0b06986b7aef7c4 WatchSource:0}: Error finding container 5ce37a9da4132b2ae206771aa4a276c64f2ee25d52d825eaa0b06986b7aef7c4: Status 404 returned error can't find the container with id 5ce37a9da4132b2ae206771aa4a276c64f2ee25d52d825eaa0b06986b7aef7c4 Jul 10 00:22:03.358501 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:03.358466 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" event={"ID":"82768138-6201-43c1-901a-915be3e69f51","Type":"ContainerStarted","Data":"5ce37a9da4132b2ae206771aa4a276c64f2ee25d52d825eaa0b06986b7aef7c4"} Jul 10 00:22:03.507933 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:03.507893 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n7czp" podUID="13e20826-95ee-455c-8405-f23540349d1e" Jul 10 00:22:04.362672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:04.362632 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" event={"ID":"82768138-6201-43c1-901a-915be3e69f51","Type":"ContainerStarted","Data":"90e0518f94b88049e272159bc36fd2f73afe3a3a0e7f640949cf58e4e001feaa"} Jul 10 00:22:04.378334 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:04.378293 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" podStartSLOduration=1.319606884 podStartE2EDuration="2.37827898s" totalImagesPullingTime="1.058672096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:04.377342292 +0000 UTC m=+296.403821007" watchObservedRunningTime="2026-07-10 00:22:04.37827898 +0000 UTC m=+296.404757684" Jul 10 00:22:05.366353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.366302 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:05.370690 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.370668 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mg6hg" Jul 10 00:22:05.462463 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.462426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:05.498935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.498901 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nxpln"] Jul 10 00:22:05.501599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.501583 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.503780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.503756 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 00:22:05.503780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.503782 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 00:22:05.503995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.503847 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 00:22:05.504062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.504010 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nqlht\"" Jul 10 00:22:05.509392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.509366 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nxpln"] Jul 10 00:22:05.529679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.529658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:05.593935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.593903 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.593935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.593936 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkvc5\" (UniqueName: \"kubernetes.io/projected/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-kube-api-access-wkvc5\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.594147 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.594030 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.594147 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.594050 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.695270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.695189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.695270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.695221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkvc5\" (UniqueName: \"kubernetes.io/projected/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-kube-api-access-wkvc5\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.695270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.695270 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.695456 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:05.695353 2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 00:22:05.695456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.695384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.695456 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:05.695413 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls podName:22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc nodeName:}" failed. No retries permitted until 2026-07-10 00:22:06.195393098 +0000 UTC m=+298.221871780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls") pod "prometheus-operator-64785b6675-nxpln" (UID: "22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc") : secret "prometheus-operator-tls" not found Jul 10 00:22:05.695916 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.695899 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.697748 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.697727 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.702497 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.702469 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkvc5\" (UniqueName: \"kubernetes.io/projected/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-kube-api-access-wkvc5\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:05.958619 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:05.958543 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:06.200164 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:06.200132 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:06.202536 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:06.202515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nxpln\" (UID: \"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:06.410717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:06.410680 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" Jul 10 00:22:06.533906 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:06.533880 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nxpln"] Jul 10 00:22:06.536129 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:06.536102 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d9d42d_ddb2_4b18_95f8_bd8a13ad4cfc.slice/crio-3af6d6b7192e49e9047e9939fbbfcc6d9cf049b253d39f8cfaf1a4b9bbd211cd WatchSource:0}: Error finding container 3af6d6b7192e49e9047e9939fbbfcc6d9cf049b253d39f8cfaf1a4b9bbd211cd: Status 404 returned error can't find the container with id 3af6d6b7192e49e9047e9939fbbfcc6d9cf049b253d39f8cfaf1a4b9bbd211cd Jul 10 00:22:07.374970 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:07.374920 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" event={"ID":"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc","Type":"ContainerStarted","Data":"3af6d6b7192e49e9047e9939fbbfcc6d9cf049b253d39f8cfaf1a4b9bbd211cd"} Jul 10 00:22:08.235147 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.235066 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:08.340681 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.340646 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4xjzv" Jul 10 00:22:08.379557 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.379526 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" event={"ID":"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc","Type":"ContainerStarted","Data":"5e6eb362bcd05e5161939a61dac20de4dcb37155ccfb77541e7c2fee147ad37b"} Jul 10 00:22:08.379728 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.379575 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" event={"ID":"22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc","Type":"ContainerStarted","Data":"6fe08cb2d8f4a6b6338d0585f0318b1d5c38aa165174074d0f60d91b061e1f7b"} Jul 10 00:22:08.403593 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.403548 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-nxpln" podStartSLOduration=1.955134919 podStartE2EDuration="3.403529184s" totalImagesPullingTime="1.448394265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:08.402416174 +0000 UTC m=+300.428894879" watchObservedRunningTime="2026-07-10 00:22:08.403529184 +0000 UTC m=+300.430007890" Jul 10 00:22:08.421020 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.420907 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:22:08.421718 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.421697 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:22:08.424547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:08.424526 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 00:22:09.598625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.598593 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:09.820509 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.820476 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:09.838324 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.838296 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5"] Jul 10 00:22:09.841326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.841311 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:09.843649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.843626 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 00:22:09.843756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.843649 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 00:22:09.843756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.843667 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wx6bh\"" Jul 10 00:22:09.851611 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.851567 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5"] Jul 10 00:22:09.856790 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.856770 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-27llx"] Jul 10 00:22:09.859739 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.859725 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.861900 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.861880 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 00:22:09.861900 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.861896 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 00:22:09.862044 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.861913 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 00:22:09.862044 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.861928 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7qsh\"" Jul 10 00:22:09.864581 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.864564 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xzj7v"] Jul 10 00:22:09.867714 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.867699 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.869690 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.869672 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 00:22:09.869799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.869712 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gszgz\"" Jul 10 00:22:09.869936 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.869920 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 00:22:09.870062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.870048 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 00:22:09.881098 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.881079 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xzj7v"] Jul 10 00:22:09.918443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.918413 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:09.928564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928533 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jgr6\" (UniqueName: \"kubernetes.io/projected/d6537777-41b8-4112-8f4e-542367b12c15-kube-api-access-4jgr6\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:09.928696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-metrics-client-ca\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.928696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bbfc9fb-45d3-4543-9757-534530eae4bd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.928696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.928696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928676 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-accelerators-collector-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928705 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928745 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928761 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-sys\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928794 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.928822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928811 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-textfile\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928843 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6537777-41b8-4112-8f4e-542367b12c15-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928909 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-wtmp\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928930 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td6pq\" (UniqueName: \"kubernetes.io/projected/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-api-access-td6pq\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-root\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.929028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.928999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:09.929206 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.929034 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrfzc\" (UniqueName: \"kubernetes.io/projected/a75c383d-36ec-4b19-84e2-038315bb0e5e-kube-api-access-lrfzc\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.929206 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.929050 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:09.970710 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:09.970679 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.029878 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.029837 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.029878 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.029876 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-textfile\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.029900 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.029926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6537777-41b8-4112-8f4e-542367b12c15-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.029980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-wtmp\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6pq\" (UniqueName: \"kubernetes.io/projected/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-api-access-td6pq\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-root\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030063 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:10.030073 2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030109 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrfzc\" (UniqueName: \"kubernetes.io/projected/a75c383d-36ec-4b19-84e2-038315bb0e5e-kube-api-access-lrfzc\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030157 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:10.030133 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls podName:a75c383d-36ec-4b19-84e2-038315bb0e5e nodeName:}" failed. No retries permitted until 2026-07-10 00:22:10.530115208 +0000 UTC m=+302.556593890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls") pod "node-exporter-27llx" (UID: "a75c383d-36ec-4b19-84e2-038315bb0e5e") : secret "node-exporter-tls" not found Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jgr6\" (UniqueName: \"kubernetes.io/projected/d6537777-41b8-4112-8f4e-542367b12c15-kube-api-access-4jgr6\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030305 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-textfile\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030313 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-metrics-client-ca\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030171 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-wtmp\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bbfc9fb-45d3-4543-9757-534530eae4bd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030388 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030413 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-accelerators-collector-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030439 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030478 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030509 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-sys\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.030639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030608 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-sys\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-metrics-client-ca\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030190 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a75c383d-36ec-4b19-84e2-038315bb0e5e-root\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030905 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6537777-41b8-4112-8f4e-542367b12c15-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:10.030930 2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.030934 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bbfc9fb-45d3-4543-9757-534530eae4bd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.031325 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:10.031007 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls podName:1bbfc9fb-45d3-4543-9757-534530eae4bd nodeName:}" failed. No retries permitted until 2026-07-10 00:22:10.530991174 +0000 UTC m=+302.557469857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-xzj7v" (UID: "1bbfc9fb-45d3-4543-9757-534530eae4bd") : secret "kube-state-metrics-tls" not found Jul 10 00:22:10.031635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.031415 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-accelerators-collector-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.031635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.031503 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.031635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.031528 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.033122 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.033098 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.033545 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.033517 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.033632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.033604 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.033920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.033899 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6537777-41b8-4112-8f4e-542367b12c15-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.041196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.041175 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrfzc\" (UniqueName: \"kubernetes.io/projected/a75c383d-36ec-4b19-84e2-038315bb0e5e-kube-api-access-lrfzc\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.041725 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.041709 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jgr6\" (UniqueName: \"kubernetes.io/projected/d6537777-41b8-4112-8f4e-542367b12c15-kube-api-access-4jgr6\") pod \"openshift-state-metrics-84bd68b896-xmvz5\" (UID: \"d6537777-41b8-4112-8f4e-542367b12c15\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.041801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.041766 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6pq\" (UniqueName: \"kubernetes.io/projected/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-api-access-td6pq\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.072702 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.072673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.150267 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.150191 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" Jul 10 00:22:10.171266 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.171241 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.271254 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.271221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.278172 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.278154 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5"] Jul 10 00:22:10.280396 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:10.280372 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6537777_41b8_4112_8f4e_542367b12c15.slice/crio-4e01d60fb2abb684dde2b3b59387c8c8462932803e88f7ef2ba80c0fb0d7bce0 WatchSource:0}: Error finding container 4e01d60fb2abb684dde2b3b59387c8c8462932803e88f7ef2ba80c0fb0d7bce0: Status 404 returned error can't find the container with id 4e01d60fb2abb684dde2b3b59387c8c8462932803e88f7ef2ba80c0fb0d7bce0 Jul 10 00:22:10.386500 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.386472 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:22:10.388304 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.388280 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" event={"ID":"d6537777-41b8-4112-8f4e-542367b12c15","Type":"ContainerStarted","Data":"6e3921c7053cdb059c99601a862312bd4d79f34d309172d47c4771e6cccb89bc"} Jul 10 00:22:10.388393 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.388356 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" event={"ID":"d6537777-41b8-4112-8f4e-542367b12c15","Type":"ContainerStarted","Data":"612fd115209a4b06dd43185b85ddfd06bc968a750c31d300b073234ada35c129"} Jul 10 00:22:10.388393 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.388376 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" event={"ID":"d6537777-41b8-4112-8f4e-542367b12c15","Type":"ContainerStarted","Data":"4e01d60fb2abb684dde2b3b59387c8c8462932803e88f7ef2ba80c0fb0d7bce0"} Jul 10 00:22:10.464825 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.464747 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.519464 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.519440 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:10.535603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.535579 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.535663 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.535614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.538077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.538055 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a75c383d-36ec-4b19-84e2-038315bb0e5e-node-exporter-tls\") pod \"node-exporter-27llx\" (UID: \"a75c383d-36ec-4b19-84e2-038315bb0e5e\") " pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.538180 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.538161 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbfc9fb-45d3-4543-9757-534530eae4bd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-xzj7v\" (UID: \"1bbfc9fb-45d3-4543-9757-534530eae4bd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.769355 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.769324 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27llx" Jul 10 00:22:10.777251 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.777102 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" Jul 10 00:22:10.788428 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:10.788388 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75c383d_36ec_4b19_84e2_038315bb0e5e.slice/crio-8952be5b3683c762df4e3e965fe57a0ebb1b96479653124e1b16bba8b60e99fb WatchSource:0}: Error finding container 8952be5b3683c762df4e3e965fe57a0ebb1b96479653124e1b16bba8b60e99fb: Status 404 returned error can't find the container with id 8952be5b3683c762df4e3e965fe57a0ebb1b96479653124e1b16bba8b60e99fb Jul 10 00:22:10.920976 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.920823 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-xzj7v"] Jul 10 00:22:10.923804 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:10.923777 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbfc9fb_45d3_4543_9757_534530eae4bd.slice/crio-a58916d4eab228fd062f051507b64cbff6cf39b2058dd0b73a9c5e61b9badc1c WatchSource:0}: Error finding container a58916d4eab228fd062f051507b64cbff6cf39b2058dd0b73a9c5e61b9badc1c: Status 404 returned error can't find the container with id a58916d4eab228fd062f051507b64cbff6cf39b2058dd0b73a9c5e61b9badc1c Jul 10 00:22:10.935679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:10.935657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:11.000335 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:11.000304 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:11.362781 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:11.362750 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:11.393981 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:11.393924 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27llx" event={"ID":"a75c383d-36ec-4b19-84e2-038315bb0e5e","Type":"ContainerStarted","Data":"8952be5b3683c762df4e3e965fe57a0ebb1b96479653124e1b16bba8b60e99fb"} Jul 10 00:22:11.395120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:11.395098 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" event={"ID":"1bbfc9fb-45d3-4543-9757-534530eae4bd","Type":"ContainerStarted","Data":"a58916d4eab228fd062f051507b64cbff6cf39b2058dd0b73a9c5e61b9badc1c"} Jul 10 00:22:12.400298 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.400263 2572 generic.go:356] "Generic (PLEG): container finished" podID="a75c383d-36ec-4b19-84e2-038315bb0e5e" containerID="d104a38f30f32f8a34744bdfbc7cc909433cc5de037f0f9167c46f0872c65512" exitCode=0 Jul 10 00:22:12.400704 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.400355 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27llx" event={"ID":"a75c383d-36ec-4b19-84e2-038315bb0e5e","Type":"ContainerDied","Data":"d104a38f30f32f8a34744bdfbc7cc909433cc5de037f0f9167c46f0872c65512"} Jul 10 00:22:12.402700 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.402679 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" event={"ID":"d6537777-41b8-4112-8f4e-542367b12c15","Type":"ContainerStarted","Data":"a323583db02440e4c12585eb61ccb1bb0a4bbeeebbe68588740c47b7dc8ea454"} Jul 10 00:22:12.433072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.433020 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xmvz5" podStartSLOduration=1.9406714840000001 podStartE2EDuration="3.433002326s" totalImagesPullingTime="1.492330842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:12.431234515 +0000 UTC m=+304.457713232" watchObservedRunningTime="2026-07-10 00:22:12.433002326 +0000 UTC m=+304.459481029" Jul 10 00:22:12.744639 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.744618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:12.818628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.818604 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7969c5-xfcls"] Jul 10 00:22:12.823392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.823366 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:12.825517 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.825499 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.835128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835101 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 00:22:12.835363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835325 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eq3n1m8up0qlq\"" Jul 10 00:22:12.835456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835368 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 00:22:12.835685 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835668 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2w7mq\"" Jul 10 00:22:12.835767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835695 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 00:22:12.835767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835679 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 00:22:12.835882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.835671 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 00:22:12.851657 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.851635 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7969c5-xfcls"] Jul 10 00:22:12.903771 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.903744 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:12.954687 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954687 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8rj4\" (UniqueName: \"kubernetes.io/projected/3b1ed516-eb97-426a-b877-fed811c75234-kube-api-access-m8rj4\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954687 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954687 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954710 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954725 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-grpc-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954740 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b1ed516-eb97-426a-b877-fed811c75234-metrics-client-ca\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:12.954926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:12.954783 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.004636 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.004606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:13.055912 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.055879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.055919 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.055942 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-grpc-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.055986 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b1ed516-eb97-426a-b877-fed811c75234-metrics-client-ca\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056360 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.056322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056480 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.056436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.056483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8rj4\" (UniqueName: \"kubernetes.io/projected/3b1ed516-eb97-426a-b877-fed811c75234-kube-api-access-m8rj4\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.056547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.056532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.057427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.056883 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b1ed516-eb97-426a-b877-fed811c75234-metrics-client-ca\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.058998 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.058926 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.059555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.059511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.059638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.059587 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.059699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.059646 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.059839 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.059799 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.059839 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.059805 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3b1ed516-eb97-426a-b877-fed811c75234-secret-grpc-tls\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.064563 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.064542 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8rj4\" (UniqueName: \"kubernetes.io/projected/3b1ed516-eb97-426a-b877-fed811c75234-kube-api-access-m8rj4\") pod \"thanos-querier-6c5b7969c5-xfcls\" (UID: \"3b1ed516-eb97-426a-b877-fed811c75234\") " pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.137225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.137193 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:13.275815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.275793 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7969c5-xfcls"] Jul 10 00:22:13.277779 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:13.277753 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1ed516_eb97_426a_b877_fed811c75234.slice/crio-89ade5c5e8bd189bd4bda6a7c168dc13849dd1c83c833d76791b5e48379307b8 WatchSource:0}: Error finding container 89ade5c5e8bd189bd4bda6a7c168dc13849dd1c83c833d76791b5e48379307b8: Status 404 returned error can't find the container with id 89ade5c5e8bd189bd4bda6a7c168dc13849dd1c83c833d76791b5e48379307b8 Jul 10 00:22:13.410994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.410939 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"89ade5c5e8bd189bd4bda6a7c168dc13849dd1c83c833d76791b5e48379307b8"} Jul 10 00:22:13.412914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.412884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" event={"ID":"1bbfc9fb-45d3-4543-9757-534530eae4bd","Type":"ContainerStarted","Data":"85549e232a3b4d3fe1d1215aed606a57fb0469fadcadee72b54e9b0d235b66c7"} Jul 10 00:22:13.413050 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.412938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" event={"ID":"1bbfc9fb-45d3-4543-9757-534530eae4bd","Type":"ContainerStarted","Data":"08df7f20da10d1a5ccb39fc122be2745759bc9d60e7954f51369cc96504c58ff"} Jul 10 00:22:13.413050 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.412979 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" event={"ID":"1bbfc9fb-45d3-4543-9757-534530eae4bd","Type":"ContainerStarted","Data":"63859c32a9a02aef01578bb1f8f7357ca4e4fa80cdb02f196dbde865648c8c83"} Jul 10 00:22:13.414803 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.414782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27llx" event={"ID":"a75c383d-36ec-4b19-84e2-038315bb0e5e","Type":"ContainerStarted","Data":"8fa9fb4a8a63dd2127a453dcf61b48a7cdd0f2f6a8c59a2fbc101cb2d998043e"} Jul 10 00:22:13.414927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.414826 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27llx" event={"ID":"a75c383d-36ec-4b19-84e2-038315bb0e5e","Type":"ContainerStarted","Data":"7991e34b6769f4f22ccd2483f8ebdd560092a33754c3ba136901dfed86aa8e7e"} Jul 10 00:22:13.429596 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.429458 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-xzj7v" podStartSLOduration=2.606926557 podStartE2EDuration="4.429443763s" totalImagesPullingTime="1.822517206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:13.429017993 +0000 UTC m=+305.455496698" watchObservedRunningTime="2026-07-10 00:22:13.429443763 +0000 UTC m=+305.455922468" Jul 10 00:22:13.445355 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:13.445310 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-27llx" podStartSLOduration=3.278771805 podStartE2EDuration="4.445296379s" totalImagesPullingTime="1.091381679s" totalInitContainerRuntime="75.142895ms" isStatefulPod=true podCreationTimestamp="2026-07-10 00:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:13.444518253 +0000 UTC m=+305.470996975" watchObservedRunningTime="2026-07-10 00:22:13.445296379 +0000 UTC m=+305.471775082" Jul 10 00:22:14.620283 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:14.620246 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:14.735172 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:14.735142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:15.317311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:15.317277 2572 patch_prober.go:28] interesting pod/image-registry-864b84b995-9x97q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 00:22:15.317483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:15.317330 2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864b84b995-9x97q" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 00:22:15.507443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:15.507402 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:22:16.038352 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:16.038257 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:16.435635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:16.435544 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"337c3034efeca2dc4b4c07db8ea76455ca4ca86e0f1daf0405fea033ed133dfc"} Jul 10 00:22:16.435635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:16.435615 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"10e0cb7a6512e8fac55e199f361e6e45b24926f97517ea14386d1b4f03c513c7"} Jul 10 00:22:16.435846 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:16.435640 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"b105e69f87e3aaba7844007f050eaf3dd3d906d8c0340ac6512f411a2f58952e"} Jul 10 00:22:17.334058 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.334034 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:22:17.441762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.441728 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"1071f384ca43d623c43e36430d33f3ebdb6efbab29e0c09b65359d579c8014b3"} Jul 10 00:22:17.441891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.441789 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"42c3ad3f6fe77cc86a97f7de42d024feb1a99a218406d7cd596b7bdbfd436371"} Jul 10 00:22:17.441891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.441816 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" event={"ID":"3b1ed516-eb97-426a-b877-fed811c75234","Type":"ContainerStarted","Data":"28314459e80dc7816f2e7f416f303f8460256d6767e969d9a268f27c2606c8dc"} Jul 10 00:22:17.441891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.441842 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:17.466888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.466838 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" podStartSLOduration=2.000139412 podStartE2EDuration="5.466822752s" totalImagesPullingTime="3.46668334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:17.465990667 +0000 UTC m=+309.492469391" watchObservedRunningTime="2026-07-10 00:22:17.466822752 +0000 UTC m=+309.493301456" Jul 10 00:22:17.565730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.565700 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:17.666874 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.666788 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:17.720491 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:17.720463 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:19.079606 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.079572 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:22:19.173036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.173000 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:19.273465 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.273430 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:19.375077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.375006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:19.632128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.632051 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:19.682427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:19.682393 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:23.452692 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:23.452660 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c5b7969c5-xfcls" Jul 10 00:22:24.210427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:24.210397 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:24.998393 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:24.998351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:25.096162 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:25.096120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:25.167199 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:25.167165 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:25.283228 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:25.283159 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:25.661223 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:25.661147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.060514 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.060461 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.116756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.116707 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.460198 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.460110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.514401 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.514372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.669925 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.669895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:26.870441 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:26.870406 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:27.439800 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:27.439769 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:28.286506 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:28.286475 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:28.335747 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:28.335717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:29.391210 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:29.391172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:29.491443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:29.491412 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:31.069126 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.069095 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:22:31.072410 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.072392 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.074842 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.074819 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 00:22:31.074980 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.074818 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4mvvr\"" Jul 10 00:22:31.075672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075654 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 00:22:31.075768 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075706 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 00:22:31.075825 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075795 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 00:22:31.075868 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075849 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 00:22:31.075919 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075858 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 00:22:31.076009 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.075988 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 00:22:31.081042 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.081020 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:22:31.206351 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206318 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.206520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206370 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.206520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206455 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqgq9\" (UniqueName: \"kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.206520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206513 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.206632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206538 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.206632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.206572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307272 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307233 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307428 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqgq9\" (UniqueName: \"kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307537 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307521 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307589 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307563 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307636 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.307861 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.307844 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.308059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.308041 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.308245 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.308229 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.310036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.310016 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.310126 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.310109 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.316291 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.316270 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqgq9\" (UniqueName: \"kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9\") pod \"console-5c65d679f9-sbwtp\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.382251 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.382174 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:31.535573 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:31.535486 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:22:31.538499 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:31.538473 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09363b72_898b_48c6_9bf1_0301e4302bf6.slice/crio-ad9b01d2b5b2d89600b99d14d8d0f685808f8add1f300824e546f49fb4e1ac7c WatchSource:0}: Error finding container ad9b01d2b5b2d89600b99d14d8d0f685808f8add1f300824e546f49fb4e1ac7c: Status 404 returned error can't find the container with id ad9b01d2b5b2d89600b99d14d8d0f685808f8add1f300824e546f49fb4e1ac7c Jul 10 00:22:32.145178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:32.145106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:32.193396 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:32.193364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:32.504301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:32.504265 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d679f9-sbwtp" event={"ID":"09363b72-898b-48c6-9bf1-0301e4302bf6","Type":"ContainerStarted","Data":"ad9b01d2b5b2d89600b99d14d8d0f685808f8add1f300824e546f49fb4e1ac7c"} Jul 10 00:22:35.519352 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:35.519317 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d679f9-sbwtp" event={"ID":"09363b72-898b-48c6-9bf1-0301e4302bf6","Type":"ContainerStarted","Data":"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c"} Jul 10 00:22:35.536803 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:35.536754 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c65d679f9-sbwtp" podStartSLOduration=1.226894851 podStartE2EDuration="4.53673893s" totalImagesPullingTime="3.309844079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:35.535525449 +0000 UTC m=+327.562004152" watchObservedRunningTime="2026-07-10 00:22:35.53673893 +0000 UTC m=+327.563217689" Jul 10 00:22:38.295301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:38.295265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:38.345967 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:38.345913 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:38.516564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:38.516525 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:38.571809 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:38.571729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:39.095738 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.095698 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:39.196646 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.196618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:39.685827 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.685797 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:22:39.692585 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.692560 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.695651 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.695627 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:22:39.701070 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.701049 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 00:22:39.782823 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782794 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.782823 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782827 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.783073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.783073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782912 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.783073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782928 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.783073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.783073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.782996 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.883945 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.883912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.883988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884024 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884039 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884069 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884328 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884140 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884742 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.884863 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884811 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.885041 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.884983 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.885154 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.885123 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.886656 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.886633 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.886802 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.886786 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:39.892373 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:39.892353 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9\") pod \"console-658846c5cd-6thmr\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:40.003402 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.003369 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:40.036901 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.036870 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:40.088622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.088595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:40.133078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.133055 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:22:40.135030 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:22:40.135001 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306c64b2_6d89_453a_8e34_0f060ae0d0dd.slice/crio-eab72e7185c170e2bc81d8f638bafb59cf4805299afc502fdaefc8e9d97e5751 WatchSource:0}: Error finding container eab72e7185c170e2bc81d8f638bafb59cf4805299afc502fdaefc8e9d97e5751: Status 404 returned error can't find the container with id eab72e7185c170e2bc81d8f638bafb59cf4805299afc502fdaefc8e9d97e5751 Jul 10 00:22:40.537838 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.537798 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658846c5cd-6thmr" event={"ID":"306c64b2-6d89-453a-8e34-0f060ae0d0dd","Type":"ContainerStarted","Data":"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff"} Jul 10 00:22:40.538026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.537850 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658846c5cd-6thmr" event={"ID":"306c64b2-6d89-453a-8e34-0f060ae0d0dd","Type":"ContainerStarted","Data":"eab72e7185c170e2bc81d8f638bafb59cf4805299afc502fdaefc8e9d97e5751"} Jul 10 00:22:40.559484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:40.559437 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658846c5cd-6thmr" podStartSLOduration=1.559424903 podStartE2EDuration="1.559424903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:22:40.557644873 +0000 UTC m=+332.584123576" watchObservedRunningTime="2026-07-10 00:22:40.559424903 +0000 UTC m=+332.585903606" Jul 10 00:22:41.383073 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:41.383038 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:41.383469 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:41.383096 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:41.387841 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:41.387815 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:41.545759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:41.545733 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:22:42.491927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:42.491894 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:42.542266 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:42.542237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:44.100111 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.100066 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864b84b995-9x97q" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerName="registry" containerID="cri-o://59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540" gracePeriod=30 Jul 10 00:22:44.296785 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.296755 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:44.346807 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.346783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:44.353224 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.353180 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:22:44.526469 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526442 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526481 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526510 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526543 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526577 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526616 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smjj9\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526663 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526718 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration\") pod \"bbf91f49-aaae-4790-855d-ec048f0a415d\" (UID: \"bbf91f49-aaae-4790-855d-ec048f0a415d\") " Jul 10 00:22:44.526909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.526868 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:22:44.527087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.527008 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-certificates\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.527087 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.527037 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:22:44.529052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.529023 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:22:44.529190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.529169 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:22:44.529379 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.529347 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9" (OuterVolumeSpecName: "kube-api-access-smjj9") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "kube-api-access-smjj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:22:44.529464 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.529380 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:22:44.529464 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.529434 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:22:44.535615 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.535593 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bbf91f49-aaae-4790-855d-ec048f0a415d" (UID: "bbf91f49-aaae-4790-855d-ec048f0a415d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:22:44.553205 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.553180 2572 generic.go:356] "Generic (PLEG): container finished" podID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerID="59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540" exitCode=0 Jul 10 00:22:44.553292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.553224 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864b84b995-9x97q" Jul 10 00:22:44.553292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.553236 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864b84b995-9x97q" event={"ID":"bbf91f49-aaae-4790-855d-ec048f0a415d","Type":"ContainerDied","Data":"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540"} Jul 10 00:22:44.553292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.553274 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864b84b995-9x97q" event={"ID":"bbf91f49-aaae-4790-855d-ec048f0a415d","Type":"ContainerDied","Data":"7e08fde3145f657a29c57750a42d3b68c2300543a09827e1f09176f0a735af72"} Jul 10 00:22:44.553408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.553305 2572 scope.go:117] "RemoveContainer" containerID="59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540" Jul 10 00:22:44.563228 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.563210 2572 scope.go:117] "RemoveContainer" containerID="59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540" Jul 10 00:22:44.563562 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:22:44.563534 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540\": container with ID starting with 59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540 not found: ID does not exist" containerID="59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540" Jul 10 00:22:44.563623 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.563573 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540"} err="failed to get container status \"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540\": rpc error: code = NotFound desc = could not find container \"59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540\": container with ID starting with 59de675416ad00dc8c738a66403e21347763873c2030d3ac7c84eda0221c8540 not found: ID does not exist" Jul 10 00:22:44.572993 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.572933 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:22:44.576636 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.576617 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-864b84b995-9x97q"] Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627637 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-bound-sa-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627660 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-image-registry-private-configuration\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627670 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf91f49-aaae-4790-855d-ec048f0a415d-installation-pull-secrets\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627680 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf91f49-aaae-4790-855d-ec048f0a415d-trusted-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627689 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf91f49-aaae-4790-855d-ec048f0a415d-ca-trust-extracted\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627696 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-registry-tls\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:44.627928 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:44.627709 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smjj9\" (UniqueName: \"kubernetes.io/projected/bbf91f49-aaae-4790-855d-ec048f0a415d-kube-api-access-smjj9\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:22:46.094876 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:46.094841 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:46.145788 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:46.145758 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:46.516480 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:46.516450 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" path="/var/lib/kubelet/pods/bbf91f49-aaae-4790-855d-ec048f0a415d/volumes" Jul 10 00:22:50.004036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.003996 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:50.004410 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.004052 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:50.008620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.008598 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:50.544357 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.544323 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:50.580340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.580320 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:22:50.622450 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:50.622418 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:22:51.493164 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:51.493121 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:51.544815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:51.544787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:52.522714 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:52.522683 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:52.573745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:52.573720 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:52.584008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:52.583977 2572 generic.go:356] "Generic (PLEG): container finished" podID="de72b07c-46c2-450d-aa9c-3c0b983e5beb" containerID="18a13e849eb7670b26356162b0c81c04759b8ffe1b7c98e1fadca656bcd5c9e9" exitCode=0 Jul 10 00:22:52.584121 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:52.584046 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" event={"ID":"de72b07c-46c2-450d-aa9c-3c0b983e5beb","Type":"ContainerDied","Data":"18a13e849eb7670b26356162b0c81c04759b8ffe1b7c98e1fadca656bcd5c9e9"} Jul 10 00:22:52.584365 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:52.584344 2572 scope.go:117] "RemoveContainer" containerID="18a13e849eb7670b26356162b0c81c04759b8ffe1b7c98e1fadca656bcd5c9e9" Jul 10 00:22:53.589391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:53.589361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-45qm7" event={"ID":"de72b07c-46c2-450d-aa9c-3c0b983e5beb","Type":"ContainerStarted","Data":"b97965530bcdfb299cb4fd3797b4aef43f57b83295d124083fcf05a989ac0628"} Jul 10 00:22:55.036405 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:55.036354 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:55.085571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:55.085541 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:22:57.605684 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:57.605649 2572 generic.go:356] "Generic (PLEG): container finished" podID="e604290b-7f97-494d-82ad-55d29971c045" containerID="b4216506689f3c98e268d5abfb8e313f6cdc82ba15eb735d73d64113107c8bb8" exitCode=0 Jul 10 00:22:57.606078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:57.605725 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" event={"ID":"e604290b-7f97-494d-82ad-55d29971c045","Type":"ContainerDied","Data":"b4216506689f3c98e268d5abfb8e313f6cdc82ba15eb735d73d64113107c8bb8"} Jul 10 00:22:57.606078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:57.606048 2572 scope.go:117] "RemoveContainer" containerID="b4216506689f3c98e268d5abfb8e313f6cdc82ba15eb735d73d64113107c8bb8" Jul 10 00:22:58.610789 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:22:58.610749 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-kz244" event={"ID":"e604290b-7f97-494d-82ad-55d29971c045","Type":"ContainerStarted","Data":"9120959622dc452a1fd4f8733e994fc959684080b0ca49da51c6c9fee4e5f8e8"} Jul 10 00:23:01.712417 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:01.712335 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:01.811016 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:01.810980 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:02.630137 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:02.630107 2572 generic.go:356] "Generic (PLEG): container finished" podID="d95732d1-a141-48f0-afff-e90eefae17d3" containerID="a6f9d5bd8af7724fcd9fb40f3864a18151eb034e564f3e2862219b65449713b2" exitCode=0 Jul 10 00:23:02.630350 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:02.630168 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" event={"ID":"d95732d1-a141-48f0-afff-e90eefae17d3","Type":"ContainerDied","Data":"a6f9d5bd8af7724fcd9fb40f3864a18151eb034e564f3e2862219b65449713b2"} Jul 10 00:23:02.630474 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:02.630462 2572 scope.go:117] "RemoveContainer" containerID="a6f9d5bd8af7724fcd9fb40f3864a18151eb034e564f3e2862219b65449713b2" Jul 10 00:23:03.636052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:03.636016 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7ppbl" event={"ID":"d95732d1-a141-48f0-afff-e90eefae17d3","Type":"ContainerStarted","Data":"d2aa5b5b5b27250679f3945172f6dfe781aa97e1efbf4060f9113d47424c05d5"} Jul 10 00:23:10.719644 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:10.719606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:10.771152 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:10.771119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:15.643765 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.643716 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c65d679f9-sbwtp" podUID="09363b72-898b-48c6-9bf1-0301e4302bf6" containerName="console" containerID="cri-o://7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c" gracePeriod=15 Jul 10 00:23:15.924293 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.924268 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:23:15.977299 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977268 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977307 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977335 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977357 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977397 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqgq9\" (UniqueName: \"kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977671 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977459 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config\") pod \"09363b72-898b-48c6-9bf1-0301e4302bf6\" (UID: \"09363b72-898b-48c6-9bf1-0301e4302bf6\") " Jul 10 00:23:15.977730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977669 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca" (OuterVolumeSpecName: "service-ca") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:23:15.977780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977731 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-service-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:15.977831 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.977795 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config" (OuterVolumeSpecName: "console-config") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:23:15.978056 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.978029 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:23:15.979783 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.979756 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:23:15.979894 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.979815 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9" (OuterVolumeSpecName: "kube-api-access-qqgq9") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "kube-api-access-qqgq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:23:15.980229 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:15.980037 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09363b72-898b-48c6-9bf1-0301e4302bf6" (UID: "09363b72-898b-48c6-9bf1-0301e4302bf6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:23:16.078525 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.078478 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-oauth-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:16.078525 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.078525 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09363b72-898b-48c6-9bf1-0301e4302bf6-console-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:16.078701 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.078540 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-console-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:16.078701 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.078554 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09363b72-898b-48c6-9bf1-0301e4302bf6-oauth-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:16.078701 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.078567 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqgq9\" (UniqueName: \"kubernetes.io/projected/09363b72-898b-48c6-9bf1-0301e4302bf6-kube-api-access-qqgq9\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:23:16.687966 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.687918 2572 generic.go:356] "Generic (PLEG): container finished" podID="09363b72-898b-48c6-9bf1-0301e4302bf6" containerID="7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c" exitCode=0 Jul 10 00:23:16.688370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.687988 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c65d679f9-sbwtp" Jul 10 00:23:16.688370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.687990 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d679f9-sbwtp" event={"ID":"09363b72-898b-48c6-9bf1-0301e4302bf6","Type":"ContainerDied","Data":"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c"} Jul 10 00:23:16.688370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.688054 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c65d679f9-sbwtp" event={"ID":"09363b72-898b-48c6-9bf1-0301e4302bf6","Type":"ContainerDied","Data":"ad9b01d2b5b2d89600b99d14d8d0f685808f8add1f300824e546f49fb4e1ac7c"} Jul 10 00:23:16.688370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.688077 2572 scope.go:117] "RemoveContainer" containerID="7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c" Jul 10 00:23:16.698422 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.698398 2572 scope.go:117] "RemoveContainer" containerID="7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c" Jul 10 00:23:16.698934 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:23:16.698909 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c\": container with ID starting with 7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c not found: ID does not exist" containerID="7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c" Jul 10 00:23:16.699059 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.698973 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c"} err="failed to get container status \"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c\": rpc error: code = NotFound desc = could not find container \"7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c\": container with ID starting with 7cc9afcc9ffc1636d91f71c36a749aaefb71c1ba657d87d1bbf47ea0d3a3424c not found: ID does not exist" Jul 10 00:23:16.704412 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.704386 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:23:16.707857 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:16.707835 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c65d679f9-sbwtp"] Jul 10 00:23:18.512209 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:18.512180 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09363b72-898b-48c6-9bf1-0301e4302bf6" path="/var/lib/kubelet/pods/09363b72-898b-48c6-9bf1-0301e4302bf6/volumes" Jul 10 00:23:22.332715 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.332679 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:23:22.335184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.335158 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13e20826-95ee-455c-8405-f23540349d1e-metrics-certs\") pod \"network-metrics-daemon-n7czp\" (UID: \"13e20826-95ee-455c-8405-f23540349d1e\") " pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:23:22.411512 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.411483 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7wp6\"" Jul 10 00:23:22.419345 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.419327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7czp" Jul 10 00:23:22.546915 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.546890 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n7czp"] Jul 10 00:23:22.549163 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:23:22.549130 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e20826_95ee_455c_8405_f23540349d1e.slice/crio-1df5b6703463feccd32b2686037fc5ea9c798fc21ff054025a2abd2fe385df55 WatchSource:0}: Error finding container 1df5b6703463feccd32b2686037fc5ea9c798fc21ff054025a2abd2fe385df55: Status 404 returned error can't find the container with id 1df5b6703463feccd32b2686037fc5ea9c798fc21ff054025a2abd2fe385df55 Jul 10 00:23:22.713315 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:22.713228 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7czp" event={"ID":"13e20826-95ee-455c-8405-f23540349d1e","Type":"ContainerStarted","Data":"1df5b6703463feccd32b2686037fc5ea9c798fc21ff054025a2abd2fe385df55"} Jul 10 00:23:24.724566 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:24.724530 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7czp" event={"ID":"13e20826-95ee-455c-8405-f23540349d1e","Type":"ContainerStarted","Data":"eddf271c2c33ad2771b0ee24cdd7e50f6a8cc523225c9d61716d47ed0aadfb8c"} Jul 10 00:23:24.724937 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:24.724582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7czp" event={"ID":"13e20826-95ee-455c-8405-f23540349d1e","Type":"ContainerStarted","Data":"4d47a9a079f43c1d63c101b1c2669c632fd03d4cf306d608368d2865bd310763"} Jul 10 00:23:24.739422 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:24.739382 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n7czp" podStartSLOduration=374.50935549 podStartE2EDuration="6m15.739368489s" totalImagesPullingTime="1.230012999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:23:24.738223617 +0000 UTC m=+376.764702332" watchObservedRunningTime="2026-07-10 00:23:24.739368489 +0000 UTC m=+376.765847193" Jul 10 00:23:24.820812 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:24.820780 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:24.876732 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:24.876703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:25.897632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:25.897599 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:25.950765 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:25.950729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:34.959691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.959653 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.959961 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerName="registry" Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.959973 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerName="registry" Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.959984 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09363b72-898b-48c6-9bf1-0301e4302bf6" containerName="console" Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.959990 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="09363b72-898b-48c6-9bf1-0301e4302bf6" containerName="console" Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.960043 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf91f49-aaae-4790-855d-ec048f0a415d" containerName="registry" Jul 10 00:23:34.960141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.960052 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="09363b72-898b-48c6-9bf1-0301e4302bf6" containerName="console" Jul 10 00:23:34.962753 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.962732 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:34.976435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:34.976413 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:23:35.013929 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.013897 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:35.044008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.043978 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scxw7\" (UniqueName: \"kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044024 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044068 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044194 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044194 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044149 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044194 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044168 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.044295 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.044194 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.114965 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.114930 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:35.145143 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145172 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145214 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145237 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145299 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145337 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scxw7\" (UniqueName: \"kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.145986 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.145939 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.146286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.146028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.146286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.146149 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.146286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.146211 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.147927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.147903 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.148052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.147978 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.154850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.154827 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scxw7\" (UniqueName: \"kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7\") pod \"console-7b6d7d697f-bp7bs\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.215030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.214927 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:35.271729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.271696 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:35.406474 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.406451 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:23:35.408793 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:23:35.408764 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9738a3_d632_4225_a99e_d0e77e8b1b00.slice/crio-1ba95bd606e09b0651c0fec72f80aec10573bcc85c70f5881be35b1a355bffcf WatchSource:0}: Error finding container 1ba95bd606e09b0651c0fec72f80aec10573bcc85c70f5881be35b1a355bffcf: Status 404 returned error can't find the container with id 1ba95bd606e09b0651c0fec72f80aec10573bcc85c70f5881be35b1a355bffcf Jul 10 00:23:35.769261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.769227 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6d7d697f-bp7bs" event={"ID":"0b9738a3-d632-4225-a99e-d0e77e8b1b00","Type":"ContainerStarted","Data":"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3"} Jul 10 00:23:35.769417 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.769281 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6d7d697f-bp7bs" event={"ID":"0b9738a3-d632-4225-a99e-d0e77e8b1b00","Type":"ContainerStarted","Data":"1ba95bd606e09b0651c0fec72f80aec10573bcc85c70f5881be35b1a355bffcf"} Jul 10 00:23:35.785173 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.785111 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b6d7d697f-bp7bs" podStartSLOduration=1.785090704 podStartE2EDuration="1.785090704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:23:35.784307616 +0000 UTC m=+387.810786320" watchObservedRunningTime="2026-07-10 00:23:35.785090704 +0000 UTC m=+387.811569409" Jul 10 00:23:35.789153 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.789125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:35.842292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:35.842257 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:37.825344 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:37.825307 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:37.923699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:37.923666 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:45.272079 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.272032 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:45.272079 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.272097 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:45.276585 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.276564 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:45.810835 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.810810 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:23:45.857705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.857667 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:23:45.954005 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:45.953972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:46.009908 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:46.009875 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:46.065753 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:46.065648 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:55.039591 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:55.039556 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:23:55.140192 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:23:55.140158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:10.885971 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:10.885883 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-658846c5cd-6thmr" podUID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" containerName="console" containerID="cri-o://2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff" gracePeriod=15 Jul 10 00:24:11.123984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.123931 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:24:11.247257 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247171 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247257 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247236 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247281 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247304 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247339 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247368 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247396 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert\") pod \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\" (UID: \"306c64b2-6d89-453a-8e34-0f060ae0d0dd\") " Jul 10 00:24:11.247714 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247647 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:24:11.247784 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247751 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config" (OuterVolumeSpecName: "console-config") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:24:11.247901 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247761 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:24:11.247968 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.247931 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:24:11.249735 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.249707 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:24:11.249838 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.249729 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9" (OuterVolumeSpecName: "kube-api-access-nm7v9") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "kube-api-access-nm7v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:24:11.249838 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.249743 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "306c64b2-6d89-453a-8e34-0f060ae0d0dd" (UID: "306c64b2-6d89-453a-8e34-0f060ae0d0dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:24:11.348282 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348252 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-oauth-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348282 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348277 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-service-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348282 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348286 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348476 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348295 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348476 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348303 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306c64b2-6d89-453a-8e34-0f060ae0d0dd-trusted-ca-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348476 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348332 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/306c64b2-6d89-453a-8e34-0f060ae0d0dd-console-oauth-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.348476 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.348341 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/306c64b2-6d89-453a-8e34-0f060ae0d0dd-kube-api-access-nm7v9\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:11.913983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.913931 2572 generic.go:356] "Generic (PLEG): container finished" podID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" containerID="2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff" exitCode=0 Jul 10 00:24:11.914451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.913989 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658846c5cd-6thmr" Jul 10 00:24:11.914451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.913999 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658846c5cd-6thmr" event={"ID":"306c64b2-6d89-453a-8e34-0f060ae0d0dd","Type":"ContainerDied","Data":"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff"} Jul 10 00:24:11.914451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.914041 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658846c5cd-6thmr" event={"ID":"306c64b2-6d89-453a-8e34-0f060ae0d0dd","Type":"ContainerDied","Data":"eab72e7185c170e2bc81d8f638bafb59cf4805299afc502fdaefc8e9d97e5751"} Jul 10 00:24:11.914451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.914060 2572 scope.go:117] "RemoveContainer" containerID="2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff" Jul 10 00:24:11.922837 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.922822 2572 scope.go:117] "RemoveContainer" containerID="2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff" Jul 10 00:24:11.923089 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:24:11.923065 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff\": container with ID starting with 2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff not found: ID does not exist" containerID="2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff" Jul 10 00:24:11.923183 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.923095 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff"} err="failed to get container status \"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff\": rpc error: code = NotFound desc = could not find container \"2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff\": container with ID starting with 2e817270cb192770c9d7b172e8c84781bb39f6cc0aea6a9ad574ac99fc8819ff not found: ID does not exist" Jul 10 00:24:11.934315 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.934295 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:24:11.938246 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:11.938227 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658846c5cd-6thmr"] Jul 10 00:24:12.512364 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:12.512332 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" path="/var/lib/kubelet/pods/306c64b2-6d89-453a-8e34-0f060ae0d0dd/volumes" Jul 10 00:24:16.200984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.200931 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:16.299841 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.299808 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:16.739530 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.739493 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv"] Jul 10 00:24:16.739910 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.739893 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" containerName="console" Jul 10 00:24:16.740032 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.739913 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" containerName="console" Jul 10 00:24:16.740032 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.740006 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="306c64b2-6d89-453a-8e34-0f060ae0d0dd" containerName="console" Jul 10 00:24:16.744372 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.744352 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.746826 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.746805 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ks6d6\"" Jul 10 00:24:16.746915 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.746892 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 00:24:16.747667 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.747655 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 00:24:16.750481 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.750462 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv"] Jul 10 00:24:16.892262 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.892229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.892455 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.892286 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9r52\" (UniqueName: \"kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.892455 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.892314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.993612 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.993517 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.993612 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.993578 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9r52\" (UniqueName: \"kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.993801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.993690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.993940 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.993921 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:16.994050 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:16.993993 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:17.001927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:17.001904 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9r52\" (UniqueName: \"kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:17.054675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:17.054640 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:17.183090 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:17.183046 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv"] Jul 10 00:24:17.185249 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:24:17.185225 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a53317_1ac8_46a3_959d_279363e3e70f.slice/crio-0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e WatchSource:0}: Error finding container 0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e: Status 404 returned error can't find the container with id 0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e Jul 10 00:24:17.940144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:17.940099 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerStarted","Data":"0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e"} Jul 10 00:24:23.971323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:23.971290 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a53317-1ac8-46a3-959d-279363e3e70f" containerID="cf1a2c3f48dda1c025fbd2b32738288413ec6a0fdc0c42738763a19fc60a5074" exitCode=0 Jul 10 00:24:23.971679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:23.971327 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerDied","Data":"cf1a2c3f48dda1c025fbd2b32738288413ec6a0fdc0c42738763a19fc60a5074"} Jul 10 00:24:26.985783 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:26.985748 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a53317-1ac8-46a3-959d-279363e3e70f" containerID="b4aed4af4930a061123118f071beee7d2e4dacf1321d4b81a1673dd5a9576b65" exitCode=0 Jul 10 00:24:26.986187 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:26.985806 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerDied","Data":"b4aed4af4930a061123118f071beee7d2e4dacf1321d4b81a1673dd5a9576b65"} Jul 10 00:24:33.484544 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:33.484461 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:34.018408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:34.018369 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerStarted","Data":"5a72115c6a55b47ff89c5754c43d32adc8ae220b747ec1ecb94671a108a71d8f"} Jul 10 00:24:34.035499 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:34.035455 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" podStartSLOduration=2.819506342 podStartE2EDuration="18.035439805s" totalImagesPullingTime="15.215933463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:24:34.033219489 +0000 UTC m=+446.059698193" watchObservedRunningTime="2026-07-10 00:24:34.035439805 +0000 UTC m=+446.061918509" Jul 10 00:24:35.023811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:35.023777 2572 generic.go:356] "Generic (PLEG): container finished" podID="20a53317-1ac8-46a3-959d-279363e3e70f" containerID="5a72115c6a55b47ff89c5754c43d32adc8ae220b747ec1ecb94671a108a71d8f" exitCode=0 Jul 10 00:24:35.024225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:35.023844 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerDied","Data":"5a72115c6a55b47ff89c5754c43d32adc8ae220b747ec1ecb94671a108a71d8f"} Jul 10 00:24:36.200574 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.200546 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:36.265966 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.265933 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util\") pod \"20a53317-1ac8-46a3-959d-279363e3e70f\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " Jul 10 00:24:36.266127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.265997 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9r52\" (UniqueName: \"kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52\") pod \"20a53317-1ac8-46a3-959d-279363e3e70f\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " Jul 10 00:24:36.266127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.266021 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle\") pod \"20a53317-1ac8-46a3-959d-279363e3e70f\" (UID: \"20a53317-1ac8-46a3-959d-279363e3e70f\") " Jul 10 00:24:36.266758 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.266728 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle" (OuterVolumeSpecName: "bundle") pod "20a53317-1ac8-46a3-959d-279363e3e70f" (UID: "20a53317-1ac8-46a3-959d-279363e3e70f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:24:36.268322 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.268302 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52" (OuterVolumeSpecName: "kube-api-access-m9r52") pod "20a53317-1ac8-46a3-959d-279363e3e70f" (UID: "20a53317-1ac8-46a3-959d-279363e3e70f"). InnerVolumeSpecName "kube-api-access-m9r52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:24:36.270360 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.270340 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util" (OuterVolumeSpecName: "util") pod "20a53317-1ac8-46a3-959d-279363e3e70f" (UID: "20a53317-1ac8-46a3-959d-279363e3e70f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:24:36.367302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.367214 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:36.367302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.367245 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9r52\" (UniqueName: \"kubernetes.io/projected/20a53317-1ac8-46a3-959d-279363e3e70f-kube-api-access-m9r52\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:36.367302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:36.367258 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20a53317-1ac8-46a3-959d-279363e3e70f-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:24:37.034909 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:37.034877 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" event={"ID":"20a53317-1ac8-46a3-959d-279363e3e70f","Type":"ContainerDied","Data":"0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e"} Jul 10 00:24:37.035097 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:37.034910 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406tfsv" Jul 10 00:24:37.035097 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:37.034945 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fc740c134b8e25f8e4ce0ab06ce9d9d9be561db71f5987b52160925c2e3c75e" Jul 10 00:24:42.757277 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:42.757220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:43.567020 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:43.566938 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:44.686858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.686821 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7"] Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687129 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="pull" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687140 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="pull" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687155 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="util" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687160 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="util" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687167 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="extract" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687173 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="extract" Jul 10 00:24:44.687235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.687224 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a53317-1ac8-46a3-959d-279363e3e70f" containerName="extract" Jul 10 00:24:44.689076 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.689061 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.691508 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.691487 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 00:24:44.691627 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.691543 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rm7nk\"" Jul 10 00:24:44.691627 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.691543 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:24:44.700259 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.700239 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7"] Jul 10 00:24:44.737153 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.737118 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhjj6\" (UniqueName: \"kubernetes.io/projected/b4111739-6b27-4507-8b47-00817f942981-kube-api-access-dhjj6\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.737300 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.737160 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4111739-6b27-4507-8b47-00817f942981-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.837711 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.837676 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhjj6\" (UniqueName: \"kubernetes.io/projected/b4111739-6b27-4507-8b47-00817f942981-kube-api-access-dhjj6\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.837880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.837717 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4111739-6b27-4507-8b47-00817f942981-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.838096 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.838076 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4111739-6b27-4507-8b47-00817f942981-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.846281 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.846259 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhjj6\" (UniqueName: \"kubernetes.io/projected/b4111739-6b27-4507-8b47-00817f942981-kube-api-access-dhjj6\") pod \"cert-manager-operator-controller-manager-55c444bf86-6lxf7\" (UID: \"b4111739-6b27-4507-8b47-00817f942981\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:44.997637 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:44.997548 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" Jul 10 00:24:45.129769 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:45.129739 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7"] Jul 10 00:24:45.133180 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:24:45.133134 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4111739_6b27_4507_8b47_00817f942981.slice/crio-8a4716462e965778833dcbb0d596ab373ad666b05008b30437eb4a3998a69a69 WatchSource:0}: Error finding container 8a4716462e965778833dcbb0d596ab373ad666b05008b30437eb4a3998a69a69: Status 404 returned error can't find the container with id 8a4716462e965778833dcbb0d596ab373ad666b05008b30437eb4a3998a69a69 Jul 10 00:24:46.073391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:46.073354 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" event={"ID":"b4111739-6b27-4507-8b47-00817f942981","Type":"ContainerStarted","Data":"8a4716462e965778833dcbb0d596ab373ad666b05008b30437eb4a3998a69a69"} Jul 10 00:24:46.385722 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:46.385636 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:46.486260 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:46.486224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:47.079706 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:47.079622 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" event={"ID":"b4111739-6b27-4507-8b47-00817f942981","Type":"ContainerStarted","Data":"119f36d61747366a19a0a5d9de29d9b006e1632a72f439f49df1b60b95675ac8"} Jul 10 00:24:47.106243 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:47.106192 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6lxf7" podStartSLOduration=1.449971273 podStartE2EDuration="3.106177603s" totalImagesPullingTime="1.65620633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:24:47.104463282 +0000 UTC m=+459.130942006" watchObservedRunningTime="2026-07-10 00:24:47.106177603 +0000 UTC m=+459.132656306" Jul 10 00:24:47.589296 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:47.589258 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:47.735034 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:47.735001 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:48.399303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.399274 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq"] Jul 10 00:24:48.401622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.401592 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.404117 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.404099 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 00:24:48.404665 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.404648 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 00:24:48.404708 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.404682 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ks6d6\"" Jul 10 00:24:48.415387 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.415364 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq"] Jul 10 00:24:48.455215 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.455179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:48.471356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.471325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.471489 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.471367 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.471489 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.471446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79nhq\" (UniqueName: \"kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.560538 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.560500 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:48.571877 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.571840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.572061 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.571882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.572061 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.571969 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79nhq\" (UniqueName: \"kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.572311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.572284 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.572311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.572303 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.580380 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.580355 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79nhq\" (UniqueName: \"kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.593386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.593361 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ldnrw"] Jul 10 00:24:48.595435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.595421 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.597762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.597744 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 00:24:48.597835 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.597823 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 00:24:48.597890 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.597841 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jlhzl\"" Jul 10 00:24:48.605365 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.605343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ldnrw"] Jul 10 00:24:48.672635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.672556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.672635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.672614 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xrxg\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-kube-api-access-9xrxg\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.711556 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.711527 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:24:48.773341 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.773305 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.773495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.773405 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xrxg\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-kube-api-access-9xrxg\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.780990 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.780966 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xrxg\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-kube-api-access-9xrxg\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.781124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.781002 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2cfab159-8313-492d-b45b-ec7bb250f479-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ldnrw\" (UID: \"2cfab159-8313-492d-b45b-ec7bb250f479\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:48.841877 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.841697 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq"] Jul 10 00:24:48.844528 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:24:48.844501 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod571278aa_0e6f_49f2_85d3_9a001f9f3ce4.slice/crio-a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80 WatchSource:0}: Error finding container a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80: Status 404 returned error can't find the container with id a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80 Jul 10 00:24:48.918437 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:48.918404 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:24:49.044265 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:49.044235 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ldnrw"] Jul 10 00:24:49.046627 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:24:49.046599 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cfab159_8313_492d_b45b_ec7bb250f479.slice/crio-537382414ad076013477a58c54db6d4128ad59df32b2d6a22b9342e18981fcee WatchSource:0}: Error finding container 537382414ad076013477a58c54db6d4128ad59df32b2d6a22b9342e18981fcee: Status 404 returned error can't find the container with id 537382414ad076013477a58c54db6d4128ad59df32b2d6a22b9342e18981fcee Jul 10 00:24:49.089438 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:49.089407 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" event={"ID":"2cfab159-8313-492d-b45b-ec7bb250f479","Type":"ContainerStarted","Data":"537382414ad076013477a58c54db6d4128ad59df32b2d6a22b9342e18981fcee"} Jul 10 00:24:49.090963 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:49.090922 2572 generic.go:356] "Generic (PLEG): container finished" podID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerID="a31b8f70b1b6883060d81c97079cb7c6acc1530ebd88d17b8002e26eb7e93a8e" exitCode=0 Jul 10 00:24:49.091076 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:49.091011 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" event={"ID":"571278aa-0e6f-49f2-85d3-9a001f9f3ce4","Type":"ContainerDied","Data":"a31b8f70b1b6883060d81c97079cb7c6acc1530ebd88d17b8002e26eb7e93a8e"} Jul 10 00:24:49.091076 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:49.091060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" event={"ID":"571278aa-0e6f-49f2-85d3-9a001f9f3ce4","Type":"ContainerStarted","Data":"a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80"} Jul 10 00:24:50.650133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:50.650105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:50.843018 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:50.842984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:53.186520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:53.186476 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:53.238086 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:53.238056 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:24:58.886833 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:24:58.886799 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:00.292723 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.292689 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n46cz"] Jul 10 00:25:00.298429 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.298409 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.300804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.300784 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mmk2z\"" Jul 10 00:25:00.305192 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.305168 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n46cz"] Jul 10 00:25:00.476075 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.476039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwrg\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-kube-api-access-4lwrg\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.476075 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.476079 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-bound-sa-token\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.577388 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.577309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwrg\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-kube-api-access-4lwrg\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.577388 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.577346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-bound-sa-token\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.585367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.585342 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwrg\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-kube-api-access-4lwrg\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.585489 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.585348 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6bd5e02-82b4-4169-9500-6226de241285-bound-sa-token\") pod \"cert-manager-55b4d954d5-n46cz\" (UID: \"d6bd5e02-82b4-4169-9500-6226de241285\") " pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.608349 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.608327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n46cz" Jul 10 00:25:00.733594 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:00.733571 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n46cz"] Jul 10 00:25:00.736210 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:25:00.736178 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6bd5e02_82b4_4169_9500_6226de241285.slice/crio-b099115cb44e5c87ea7745150b65869c140771f750fb39450f44c03714fd16eb WatchSource:0}: Error finding container b099115cb44e5c87ea7745150b65869c140771f750fb39450f44c03714fd16eb: Status 404 returned error can't find the container with id b099115cb44e5c87ea7745150b65869c140771f750fb39450f44c03714fd16eb Jul 10 00:25:01.144797 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:01.144764 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n46cz" event={"ID":"d6bd5e02-82b4-4169-9500-6226de241285","Type":"ContainerStarted","Data":"b099115cb44e5c87ea7745150b65869c140771f750fb39450f44c03714fd16eb"} Jul 10 00:25:04.163190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.163086 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" event={"ID":"2cfab159-8313-492d-b45b-ec7bb250f479","Type":"ContainerStarted","Data":"df0cdcfaf6513d60b75bbfe49c5b51e6927c71e1f9a6863b66a97701cd05c936"} Jul 10 00:25:04.163190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.163143 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:25:04.164680 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.164651 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n46cz" event={"ID":"d6bd5e02-82b4-4169-9500-6226de241285","Type":"ContainerStarted","Data":"bc1a6814fd2b77311abd18778392ea66532aad253809699a0c5f0a2244ed50ad"} Jul 10 00:25:04.166532 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.166509 2572 generic.go:356] "Generic (PLEG): container finished" podID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerID="3f8f7b606307cd6d86f8c5801118a6c17b1dfcc31238630175c8f53f3931dd58" exitCode=0 Jul 10 00:25:04.166685 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.166555 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" event={"ID":"571278aa-0e6f-49f2-85d3-9a001f9f3ce4","Type":"ContainerDied","Data":"3f8f7b606307cd6d86f8c5801118a6c17b1dfcc31238630175c8f53f3931dd58"} Jul 10 00:25:04.167405 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.167386 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:04.181053 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.181014 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" podStartSLOduration=1.3349823299999999 podStartE2EDuration="16.180999724s" totalImagesPullingTime="14.846017394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:25:04.179441656 +0000 UTC m=+476.205920359" watchObservedRunningTime="2026-07-10 00:25:04.180999724 +0000 UTC m=+476.207478429" Jul 10 00:25:04.209353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.209314 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n46cz" podStartSLOduration=1.046544819 podStartE2EDuration="4.209298821s" totalImagesPullingTime="3.162754002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:25:04.208129082 +0000 UTC m=+476.234607786" watchObservedRunningTime="2026-07-10 00:25:04.209298821 +0000 UTC m=+476.235777525" Jul 10 00:25:04.267751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:04.267722 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:05.175447 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:05.175372 2572 generic.go:356] "Generic (PLEG): container finished" podID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerID="7f59ef3fdcaf7d3f21a0471074795743ea3dd31a8aec2fe25adda81ef61d339a" exitCode=0 Jul 10 00:25:05.175886 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:05.175455 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" event={"ID":"571278aa-0e6f-49f2-85d3-9a001f9f3ce4","Type":"ContainerDied","Data":"7f59ef3fdcaf7d3f21a0471074795743ea3dd31a8aec2fe25adda81ef61d339a"} Jul 10 00:25:06.318710 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.318684 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:25:06.430106 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.430080 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util\") pod \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " Jul 10 00:25:06.430290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.430164 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79nhq\" (UniqueName: \"kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq\") pod \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " Jul 10 00:25:06.430290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.430217 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle\") pod \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\" (UID: \"571278aa-0e6f-49f2-85d3-9a001f9f3ce4\") " Jul 10 00:25:06.430624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.430601 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle" (OuterVolumeSpecName: "bundle") pod "571278aa-0e6f-49f2-85d3-9a001f9f3ce4" (UID: "571278aa-0e6f-49f2-85d3-9a001f9f3ce4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:25:06.432392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.432342 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq" (OuterVolumeSpecName: "kube-api-access-79nhq") pod "571278aa-0e6f-49f2-85d3-9a001f9f3ce4" (UID: "571278aa-0e6f-49f2-85d3-9a001f9f3ce4"). InnerVolumeSpecName "kube-api-access-79nhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:25:06.434931 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.434907 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util" (OuterVolumeSpecName: "util") pod "571278aa-0e6f-49f2-85d3-9a001f9f3ce4" (UID: "571278aa-0e6f-49f2-85d3-9a001f9f3ce4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:25:06.531666 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.531640 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79nhq\" (UniqueName: \"kubernetes.io/projected/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-kube-api-access-79nhq\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:06.531666 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.531663 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:06.531806 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:06.531673 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/571278aa-0e6f-49f2-85d3-9a001f9f3ce4-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:07.187820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:07.187796 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" Jul 10 00:25:07.188008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:07.187795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frrdrq" event={"ID":"571278aa-0e6f-49f2-85d3-9a001f9f3ce4","Type":"ContainerDied","Data":"a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80"} Jul 10 00:25:07.188008 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:07.187909 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a75e1703973ee3422e1abcb86de4a48185ba4f8fc173931275d8acf0751d1d80" Jul 10 00:25:10.178263 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:10.178234 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ldnrw" Jul 10 00:25:10.692012 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:10.691979 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:10.792691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:10.792660 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:12.516804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.516777 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk"] Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517087 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="pull" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517098 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="pull" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517111 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="util" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517116 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="util" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517124 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="extract" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517129 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="extract" Jul 10 00:25:12.517196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.517188 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="571278aa-0e6f-49f2-85d3-9a001f9f3ce4" containerName="extract" Jul 10 00:25:12.519306 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.519290 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.521531 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.521504 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:25:12.521531 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.521524 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-925qv\"" Jul 10 00:25:12.521687 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.521646 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 00:25:12.527396 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.527375 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk"] Jul 10 00:25:12.581613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.581533 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02eb94e2-e914-4aed-9192-2e7fa6cb2356-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.581613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.581598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9nvz\" (UniqueName: \"kubernetes.io/projected/02eb94e2-e914-4aed-9192-2e7fa6cb2356-kube-api-access-n9nvz\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.682873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.682842 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02eb94e2-e914-4aed-9192-2e7fa6cb2356-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.683052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.682895 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9nvz\" (UniqueName: \"kubernetes.io/projected/02eb94e2-e914-4aed-9192-2e7fa6cb2356-kube-api-access-n9nvz\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.683246 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.683227 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02eb94e2-e914-4aed-9192-2e7fa6cb2356-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.693069 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.693045 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9nvz\" (UniqueName: \"kubernetes.io/projected/02eb94e2-e914-4aed-9192-2e7fa6cb2356-kube-api-access-n9nvz\") pod \"openshift-lws-operator-fd8ccff4c-4hmxk\" (UID: \"02eb94e2-e914-4aed-9192-2e7fa6cb2356\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:12.828640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:12.828608 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" Jul 10 00:25:13.001108 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:13.001075 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk"] Jul 10 00:25:13.003555 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:25:13.003526 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02eb94e2_e914_4aed_9192_2e7fa6cb2356.slice/crio-7f82a76c86dd466a2c8303d834367d76efe52b5b676717ab52fdc130ccdb5fb8 WatchSource:0}: Error finding container 7f82a76c86dd466a2c8303d834367d76efe52b5b676717ab52fdc130ccdb5fb8: Status 404 returned error can't find the container with id 7f82a76c86dd466a2c8303d834367d76efe52b5b676717ab52fdc130ccdb5fb8 Jul 10 00:25:13.216281 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:13.216203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" event={"ID":"02eb94e2-e914-4aed-9192-2e7fa6cb2356","Type":"ContainerStarted","Data":"7f82a76c86dd466a2c8303d834367d76efe52b5b676717ab52fdc130ccdb5fb8"} Jul 10 00:25:15.228325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:15.228231 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" event={"ID":"02eb94e2-e914-4aed-9192-2e7fa6cb2356","Type":"ContainerStarted","Data":"86bf5d59a69ea9c544f26a0800c0554374d2f4ec591a9451f9d0d25a2979bc2d"} Jul 10 00:25:15.242643 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:15.242593 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4hmxk" podStartSLOduration=1.268913389 podStartE2EDuration="3.242574624s" totalImagesPullingTime="1.973661235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:25:15.242040598 +0000 UTC m=+487.268519327" watchObservedRunningTime="2026-07-10 00:25:15.242574624 +0000 UTC m=+487.269053324" Jul 10 00:25:16.079192 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:16.079161 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:16.179155 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:16.179122 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:17.955086 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.955052 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg"] Jul 10 00:25:17.957433 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.957415 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:17.959603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.959580 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ks6d6\"" Jul 10 00:25:17.959742 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.959729 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 00:25:17.960427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.960410 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 00:25:17.965102 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:17.965078 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg"] Jul 10 00:25:18.031287 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.031250 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.031462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.031297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpklv\" (UniqueName: \"kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.031462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.031377 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.132225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.132194 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.132381 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.132271 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.132381 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.132296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpklv\" (UniqueName: \"kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.132623 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.132600 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.132658 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.132609 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.140195 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.140169 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpklv\" (UniqueName: \"kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.267555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.267524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:18.268544 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.268517 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:18.397984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:18.397942 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg"] Jul 10 00:25:18.399917 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:25:18.399890 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f039db_9c7f_408d_b75e_7d3dcb6a1cc1.slice/crio-bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107 WatchSource:0}: Error finding container bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107: Status 404 returned error can't find the container with id bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107 Jul 10 00:25:19.246858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:19.246823 2572 generic.go:356] "Generic (PLEG): container finished" podID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerID="75185a7d6a8a29174a91833410259f67e96f96fc14e5468727d580ac0b226e4b" exitCode=0 Jul 10 00:25:19.247218 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:19.246864 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" event={"ID":"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1","Type":"ContainerDied","Data":"75185a7d6a8a29174a91833410259f67e96f96fc14e5468727d580ac0b226e4b"} Jul 10 00:25:19.247218 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:19.246907 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" event={"ID":"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1","Type":"ContainerStarted","Data":"bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107"} Jul 10 00:25:20.258201 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:20.258169 2572 generic.go:356] "Generic (PLEG): container finished" podID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerID="e2533e2e1a5f199ae848deed1d46969401eb187eaed32fa2ebf31c4b5ffa741c" exitCode=0 Jul 10 00:25:20.258608 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:20.258258 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" event={"ID":"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1","Type":"ContainerDied","Data":"e2533e2e1a5f199ae848deed1d46969401eb187eaed32fa2ebf31c4b5ffa741c"} Jul 10 00:25:21.264882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:21.264851 2572 generic.go:356] "Generic (PLEG): container finished" podID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerID="98f245c537955fb1b125e944ea040b9f0330b662901c3685eba9258b909fc43b" exitCode=0 Jul 10 00:25:21.264882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:21.264884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" event={"ID":"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1","Type":"ContainerDied","Data":"98f245c537955fb1b125e944ea040b9f0330b662901c3685eba9258b909fc43b"} Jul 10 00:25:22.397583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.397556 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:22.472133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.472097 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util\") pod \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " Jul 10 00:25:22.472133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.472140 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle\") pod \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " Jul 10 00:25:22.472341 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.472193 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpklv\" (UniqueName: \"kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv\") pod \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\" (UID: \"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1\") " Jul 10 00:25:22.472939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.472906 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle" (OuterVolumeSpecName: "bundle") pod "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" (UID: "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:25:22.474371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.474348 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv" (OuterVolumeSpecName: "kube-api-access-xpklv") pod "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" (UID: "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1"). InnerVolumeSpecName "kube-api-access-xpklv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:25:22.481153 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.481117 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util" (OuterVolumeSpecName: "util") pod "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" (UID: "56f039db-9c7f-408d-b75e-7d3dcb6a1cc1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:25:22.573371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.573306 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:22.573371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.573329 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:22.573371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:22.573337 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpklv\" (UniqueName: \"kubernetes.io/projected/56f039db-9c7f-408d-b75e-7d3dcb6a1cc1-kube-api-access-xpklv\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:25:23.276400 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:23.276362 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" event={"ID":"56f039db-9c7f-408d-b75e-7d3dcb6a1cc1","Type":"ContainerDied","Data":"bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107"} Jul 10 00:25:23.276400 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:23.276408 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb2bda7cdb8646a4a8ca554a648f430b1f85920082f5ea61d4270427879a4107" Jul 10 00:25:23.276605 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:23.276379 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5jgg" Jul 10 00:25:26.498641 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:26.498610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:26.572530 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:26.572491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:42.032176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:42.032119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:42.082926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:42.082899 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:45.242138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:45.242107 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:45.341258 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:45.341225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:45.852134 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:45.852103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.052144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.052110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.151351 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.151262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.184270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184236 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r"] Jul 10 00:25:46.184597 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184584 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="extract" Jul 10 00:25:46.184640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184599 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="extract" Jul 10 00:25:46.184640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184608 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="util" Jul 10 00:25:46.184640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184613 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="util" Jul 10 00:25:46.184640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184625 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="pull" Jul 10 00:25:46.184640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184631 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="pull" Jul 10 00:25:46.184783 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.184683 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="56f039db-9c7f-408d-b75e-7d3dcb6a1cc1" containerName="extract" Jul 10 00:25:46.192751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.192729 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.195140 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.195114 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 00:25:46.195273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.195188 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 00:25:46.195273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.195228 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 00:25:46.195273 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.195266 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jktj7\"" Jul 10 00:25:46.195494 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.195481 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 00:25:46.205392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.205370 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r"] Jul 10 00:25:46.276680 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.276646 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-manifests\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.277204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.276688 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxzbz\" (UniqueName: \"kubernetes.io/projected/66e5bf26-c5c4-4f07-9d75-9693d63b665c-kube-api-access-lxzbz\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.277204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.276737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.277204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.276833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.277204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.276877 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-tmp\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.377573 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.377541 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.377763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.377592 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-tmp\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.377763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.377647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-manifests\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.377763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.377668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzbz\" (UniqueName: \"kubernetes.io/projected/66e5bf26-c5c4-4f07-9d75-9693d63b665c-kube-api-access-lxzbz\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.377763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.377710 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.378221 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.378188 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-manifests\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.380074 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.380050 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e5bf26-c5c4-4f07-9d75-9693d63b665c-tmp\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.380442 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.380422 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.380532 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.380511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e5bf26-c5c4-4f07-9d75-9693d63b665c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.386208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.386179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.386606 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.386586 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxzbz\" (UniqueName: \"kubernetes.io/projected/66e5bf26-c5c4-4f07-9d75-9693d63b665c-kube-api-access-lxzbz\") pod \"opendatahub-operator-controller-manager-85d875657f-52h9r\" (UID: \"66e5bf26-c5c4-4f07-9d75-9693d63b665c\") " pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.487761 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.487686 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.503860 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.503827 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:46.590345 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.590309 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.639991 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.639938 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r"] Jul 10 00:25:46.642163 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:25:46.642133 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e5bf26_c5c4_4f07_9d75_9693d63b665c.slice/crio-531a4a74b3c67795b4a33844d6346f52adb11494a8ca98cc9e152c595371ad59 WatchSource:0}: Error finding container 531a4a74b3c67795b4a33844d6346f52adb11494a8ca98cc9e152c595371ad59: Status 404 returned error can't find the container with id 531a4a74b3c67795b4a33844d6346f52adb11494a8ca98cc9e152c595371ad59 Jul 10 00:25:46.688181 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.688148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.788646 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.788618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:46.893971 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:46.890515 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:25:47.389935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:47.389818 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" event={"ID":"66e5bf26-c5c4-4f07-9d75-9693d63b665c","Type":"ContainerStarted","Data":"531a4a74b3c67795b4a33844d6346f52adb11494a8ca98cc9e152c595371ad59"} Jul 10 00:25:50.408031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:50.407994 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" event={"ID":"66e5bf26-c5c4-4f07-9d75-9693d63b665c","Type":"ContainerStarted","Data":"043092b02b5a55aef85daae2fa8ef4a7284cf5d594831a205f72bbd815faa8a2"} Jul 10 00:25:51.413829 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:51.413786 2572 generic.go:356] "Generic (PLEG): container finished" podID="66e5bf26-c5c4-4f07-9d75-9693d63b665c" containerID="043092b02b5a55aef85daae2fa8ef4a7284cf5d594831a205f72bbd815faa8a2" exitCode=0 Jul 10 00:25:51.414298 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:51.413881 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" event={"ID":"66e5bf26-c5c4-4f07-9d75-9693d63b665c","Type":"ContainerDied","Data":"043092b02b5a55aef85daae2fa8ef4a7284cf5d594831a205f72bbd815faa8a2"} Jul 10 00:25:52.421019 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:52.420980 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" event={"ID":"66e5bf26-c5c4-4f07-9d75-9693d63b665c","Type":"ContainerStarted","Data":"948bedd01f62741e0daa003ed054fd463fd3d29be370d119a9003c748d1b81b0"} Jul 10 00:25:52.421499 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:52.421061 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:25:52.441322 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:25:52.441279 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" podStartSLOduration=2.007591209 podStartE2EDuration="6.441264335s" totalImagesPullingTime="3.403827843s" totalInitContainerRuntime="1.029845283s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:25:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:25:52.439633897 +0000 UTC m=+524.466112600" watchObservedRunningTime="2026-07-10 00:25:52.441264335 +0000 UTC m=+524.467743039" Jul 10 00:26:03.428286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:03.428194 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85d875657f-52h9r" Jul 10 00:26:05.193126 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.193087 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:05.292334 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.292292 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:05.804658 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.804626 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-69dn5"] Jul 10 00:26:05.808125 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.808106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.810480 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.810455 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 00:26:05.810595 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.810461 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-767xm\"" Jul 10 00:26:05.819103 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.819078 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-69dn5"] Jul 10 00:26:05.833340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.833314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:05.840862 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.840840 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76c5e371-465f-48f4-951d-cc695e6eaa9d-cert\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.840973 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.840881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xlr\" (UniqueName: \"kubernetes.io/projected/76c5e371-465f-48f4-951d-cc695e6eaa9d-kube-api-access-g6xlr\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.933336 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.933303 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:05.941471 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.941447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xlr\" (UniqueName: \"kubernetes.io/projected/76c5e371-465f-48f4-951d-cc695e6eaa9d-kube-api-access-g6xlr\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.941570 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.941528 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76c5e371-465f-48f4-951d-cc695e6eaa9d-cert\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.944055 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.944029 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76c5e371-465f-48f4-951d-cc695e6eaa9d-cert\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:05.949599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:05.949567 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xlr\" (UniqueName: \"kubernetes.io/projected/76c5e371-465f-48f4-951d-cc695e6eaa9d-kube-api-access-g6xlr\") pod \"odh-model-controller-7998c67dc4-69dn5\" (UID: \"76c5e371-465f-48f4-951d-cc695e6eaa9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:06.034692 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.034659 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:06.120915 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.120836 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:06.248303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.248281 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-69dn5"] Jul 10 00:26:06.250585 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:06.250560 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c5e371_465f_48f4_951d_cc695e6eaa9d.slice/crio-eadf86df6ffc0c510551053d4dec8cee5f14ad9ab3902a4c1940657489578063 WatchSource:0}: Error finding container eadf86df6ffc0c510551053d4dec8cee5f14ad9ab3902a4c1940657489578063: Status 404 returned error can't find the container with id eadf86df6ffc0c510551053d4dec8cee5f14ad9ab3902a4c1940657489578063 Jul 10 00:26:06.311824 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.311785 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjx67"] Jul 10 00:26:06.316389 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.316370 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.318525 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.318506 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 00:26:06.318622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.318577 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rgll6\"" Jul 10 00:26:06.322081 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.322062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:06.326846 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.326822 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjx67"] Jul 10 00:26:06.345693 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.345666 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6z9\" (UniqueName: \"kubernetes.io/projected/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-kube-api-access-pd6z9\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.345693 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.345696 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.446187 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.446108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6z9\" (UniqueName: \"kubernetes.io/projected/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-kube-api-access-pd6z9\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.446187 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.446143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.446374 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:06.446264 2572 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 00:26:06.446374 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:06.446336 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs podName:7a10a9e2-b0d2-4377-a722-90bbf2dd5929 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:06.946315768 +0000 UTC m=+538.972794467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs") pod "model-serving-api-54d89bd5fc-qjx67" (UID: "7a10a9e2-b0d2-4377-a722-90bbf2dd5929") : secret "model-serving-api-tls" not found Jul 10 00:26:06.454672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.454649 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6z9\" (UniqueName: \"kubernetes.io/projected/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-kube-api-access-pd6z9\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.487330 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.487305 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" event={"ID":"76c5e371-465f-48f4-951d-cc695e6eaa9d","Type":"ContainerStarted","Data":"eadf86df6ffc0c510551053d4dec8cee5f14ad9ab3902a4c1940657489578063"} Jul 10 00:26:06.532029 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.531995 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:06.951171 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.951084 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:06.955325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:06.955293 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a10a9e2-b0d2-4377-a722-90bbf2dd5929-tls-certs\") pod \"model-serving-api-54d89bd5fc-qjx67\" (UID: \"7a10a9e2-b0d2-4377-a722-90bbf2dd5929\") " pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:07.228920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:07.228833 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:07.369332 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:07.369218 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qjx67"] Jul 10 00:26:07.372440 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:07.372401 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a10a9e2_b0d2_4377_a722_90bbf2dd5929.slice/crio-ae3c9b2809f6006cf93580fcb3e5ed0f2794b6b8a0994252dd179001c37e122c WatchSource:0}: Error finding container ae3c9b2809f6006cf93580fcb3e5ed0f2794b6b8a0994252dd179001c37e122c: Status 404 returned error can't find the container with id ae3c9b2809f6006cf93580fcb3e5ed0f2794b6b8a0994252dd179001c37e122c Jul 10 00:26:07.495796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:07.495707 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" event={"ID":"7a10a9e2-b0d2-4377-a722-90bbf2dd5929","Type":"ContainerStarted","Data":"ae3c9b2809f6006cf93580fcb3e5ed0f2794b6b8a0994252dd179001c37e122c"} Jul 10 00:26:09.331202 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:09.331156 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:09.431565 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:09.431533 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:10.513211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.513179 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" event={"ID":"76c5e371-465f-48f4-951d-cc695e6eaa9d","Type":"ContainerStarted","Data":"d7954f0b190709468c3df1536c94c34775a7580531db62f3d7f9664b7b64bb57"} Jul 10 00:26:10.513625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.513242 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:10.513625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.513517 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" event={"ID":"7a10a9e2-b0d2-4377-a722-90bbf2dd5929","Type":"ContainerStarted","Data":"98832e0ed9dcf5a381983d7ef5fa84229635a9b7d6c1e08b5c4a1958f79475f6"} Jul 10 00:26:10.513625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.513565 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:10.529855 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.529812 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" podStartSLOduration=1.814418292 podStartE2EDuration="5.529796976s" totalImagesPullingTime="3.715378684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:10.529040774 +0000 UTC m=+542.555519490" watchObservedRunningTime="2026-07-10 00:26:10.529796976 +0000 UTC m=+542.556275683" Jul 10 00:26:10.549925 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:10.549881 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" podStartSLOduration=1.962070237 podStartE2EDuration="4.549865964s" totalImagesPullingTime="2.587795727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:10.548552084 +0000 UTC m=+542.575030788" watchObservedRunningTime="2026-07-10 00:26:10.549865964 +0000 UTC m=+542.576344668" Jul 10 00:26:14.660914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.660881 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:14.698615 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.698585 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk"] Jul 10 00:26:14.702027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.702008 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:14.704865 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.704846 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 00:26:14.704865 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.704858 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-pzpkg\"" Jul 10 00:26:14.711474 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.711449 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk"] Jul 10 00:26:14.714148 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.714129 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7"] Jul 10 00:26:14.717497 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.717479 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.720259 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.720240 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dqlzv\"" Jul 10 00:26:14.720357 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.720277 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 00:26:14.733263 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.733237 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7"] Jul 10 00:26:14.738640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.738617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:14.822184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.822151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:14.822356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.822192 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.822356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.822255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzb6h\" (UniqueName: \"kubernetes.io/projected/8dfe28e5-4518-4289-a6a9-f4898c92c92f-kube-api-access-tzb6h\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.822356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.822297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24r2d\" (UniqueName: \"kubernetes.io/projected/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-kube-api-access-24r2d\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:14.922749 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.922667 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.922749 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.922716 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzb6h\" (UniqueName: \"kubernetes.io/projected/8dfe28e5-4518-4289-a6a9-f4898c92c92f-kube-api-access-tzb6h\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.922749 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.922741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24r2d\" (UniqueName: \"kubernetes.io/projected/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-kube-api-access-24r2d\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:14.923048 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.922786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:14.923048 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:14.922831 2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 00:26:14.923048 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:14.922867 2572 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 00:26:14.923048 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:14.922917 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert podName:c2eaf9cc-d308-43a0-8cca-35f16e605aeb nodeName:}" failed. No retries permitted until 2026-07-10 00:26:15.422901734 +0000 UTC m=+547.449380416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert") pod "kserve-controller-manager-d6f4b65b8-gzbxk" (UID: "c2eaf9cc-d308-43a0-8cca-35f16e605aeb") : secret "kserve-webhook-server-cert" not found Jul 10 00:26:14.923048 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:26:14.922931 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert podName:8dfe28e5-4518-4289-a6a9-f4898c92c92f nodeName:}" failed. No retries permitted until 2026-07-10 00:26:15.422922705 +0000 UTC m=+547.449401387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert") pod "llmisvc-controller-manager-777c6bd7d5-zl9h7" (UID: "8dfe28e5-4518-4289-a6a9-f4898c92c92f") : secret "llmisvc-webhook-server-cert" not found Jul 10 00:26:14.931880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.931859 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzb6h\" (UniqueName: \"kubernetes.io/projected/8dfe28e5-4518-4289-a6a9-f4898c92c92f-kube-api-access-tzb6h\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:14.932030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:14.931904 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24r2d\" (UniqueName: \"kubernetes.io/projected/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-kube-api-access-24r2d\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:15.016654 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.016621 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:15.428011 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.427982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:15.428253 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.428023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:15.430699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.430672 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfe28e5-4518-4289-a6a9-f4898c92c92f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zl9h7\" (UID: \"8dfe28e5-4518-4289-a6a9-f4898c92c92f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:15.430812 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.430711 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2eaf9cc-d308-43a0-8cca-35f16e605aeb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzbxk\" (UID: \"c2eaf9cc-d308-43a0-8cca-35f16e605aeb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:15.448172 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.448111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:15.613048 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.613022 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:15.626980 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.626938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:15.758984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.758935 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk"] Jul 10 00:26:15.761392 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:15.761324 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2eaf9cc_d308_43a0_8cca_35f16e605aeb.slice/crio-d6b2982cdbc801c9f86ab6353504b03029ade3398b3dc510b74e3d352c429eb3 WatchSource:0}: Error finding container d6b2982cdbc801c9f86ab6353504b03029ade3398b3dc510b74e3d352c429eb3: Status 404 returned error can't find the container with id d6b2982cdbc801c9f86ab6353504b03029ade3398b3dc510b74e3d352c429eb3 Jul 10 00:26:15.779888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.779863 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7"] Jul 10 00:26:15.781506 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:15.781479 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8dfe28e5_4518_4289_a6a9_f4898c92c92f.slice/crio-ae6517f62757aaebdbacd2fd32a50b294fd239686faaa9d7e9edca26a9a3301b WatchSource:0}: Error finding container ae6517f62757aaebdbacd2fd32a50b294fd239686faaa9d7e9edca26a9a3301b: Status 404 returned error can't find the container with id ae6517f62757aaebdbacd2fd32a50b294fd239686faaa9d7e9edca26a9a3301b Jul 10 00:26:15.931864 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.929546 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:15.979917 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.979833 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4"] Jul 10 00:26:15.985020 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.984996 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:15.987478 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.987446 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 00:26:15.987594 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.987452 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 00:26:15.987594 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.987512 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mzggw\"" Jul 10 00:26:15.993675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:15.993648 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4"] Jul 10 00:26:16.015239 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.015209 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:16.037729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.037695 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tls-certs\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.037890 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.037773 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tmp\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.037890 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.037805 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rq26\" (UniqueName: \"kubernetes.io/projected/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-kube-api-access-4rq26\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.138985 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.138934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tmp\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.139165 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.139003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rq26\" (UniqueName: \"kubernetes.io/projected/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-kube-api-access-4rq26\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.139165 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.139061 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tls-certs\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.141605 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.141577 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tmp\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.141770 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.141755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-tls-certs\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.156795 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.156769 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rq26\" (UniqueName: \"kubernetes.io/projected/0721fc09-8f0a-4f00-82e9-9b5fd08171dc-kube-api-access-4rq26\") pod \"kube-auth-proxy-77865f7b8-lz6q4\" (UID: \"0721fc09-8f0a-4f00-82e9-9b5fd08171dc\") " pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.159858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.159837 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:16.244299 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.244208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:16.299250 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.299211 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" Jul 10 00:26:16.433098 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.433072 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4"] Jul 10 00:26:16.435282 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:16.435256 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0721fc09_8f0a_4f00_82e9_9b5fd08171dc.slice/crio-15cf4054c3b10b9aab05dbc637ea31435e02a8d7e3816c30e3012d75ed59b596 WatchSource:0}: Error finding container 15cf4054c3b10b9aab05dbc637ea31435e02a8d7e3816c30e3012d75ed59b596: Status 404 returned error can't find the container with id 15cf4054c3b10b9aab05dbc637ea31435e02a8d7e3816c30e3012d75ed59b596 Jul 10 00:26:16.547793 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.547721 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" event={"ID":"0721fc09-8f0a-4f00-82e9-9b5fd08171dc","Type":"ContainerStarted","Data":"15cf4054c3b10b9aab05dbc637ea31435e02a8d7e3816c30e3012d75ed59b596"} Jul 10 00:26:16.549170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.549134 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" event={"ID":"8dfe28e5-4518-4289-a6a9-f4898c92c92f","Type":"ContainerStarted","Data":"ae6517f62757aaebdbacd2fd32a50b294fd239686faaa9d7e9edca26a9a3301b"} Jul 10 00:26:16.550648 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:16.550622 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" event={"ID":"c2eaf9cc-d308-43a0-8cca-35f16e605aeb","Type":"ContainerStarted","Data":"d6b2982cdbc801c9f86ab6353504b03029ade3398b3dc510b74e3d352c429eb3"} Jul 10 00:26:17.628670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.628632 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:17.632916 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.632886 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh"] Jul 10 00:26:17.636652 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.636632 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.639208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.639182 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 00:26:17.639317 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.639236 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 00:26:17.639387 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.639328 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qzk9h\"" Jul 10 00:26:17.639486 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.639470 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 00:26:17.639538 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.639482 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 00:26:17.647983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.647934 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh"] Jul 10 00:26:17.731371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.731334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:17.754967 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.754924 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.754990 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.755017 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpb2d\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-kube-api-access-mpb2d\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.755047 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa9c659-33bc-4916-b33f-c1865ee56fe3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755120 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.755091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755350 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.755122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.755350 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.755150 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856215 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856215 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856217 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856273 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpb2d\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-kube-api-access-mpb2d\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856330 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa9c659-33bc-4916-b33f-c1865ee56fe3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856446 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.856823 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.856798 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.858826 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.858804 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.858971 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.858935 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.859848 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.859831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fa9c659-33bc-4916-b33f-c1865ee56fe3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.859966 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.859931 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.874975 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.874933 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.875178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.875163 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpb2d\" (UniqueName: \"kubernetes.io/projected/0fa9c659-33bc-4916-b33f-c1865ee56fe3-kube-api-access-mpb2d\") pod \"istiod-openshift-gateway-94bb8fbfd-4l2fh\" (UID: \"0fa9c659-33bc-4916-b33f-c1865ee56fe3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:17.946326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:17.946142 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:18.106007 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:18.105920 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh"] Jul 10 00:26:18.108512 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:26:18.108481 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa9c659_33bc_4916_b33f_c1865ee56fe3.slice/crio-5fad2ecab2f3a140aa82b863aebe7bfa8782d675bdeaca7dba896a1757fb9b2e WatchSource:0}: Error finding container 5fad2ecab2f3a140aa82b863aebe7bfa8782d675bdeaca7dba896a1757fb9b2e: Status 404 returned error can't find the container with id 5fad2ecab2f3a140aa82b863aebe7bfa8782d675bdeaca7dba896a1757fb9b2e Jul 10 00:26:18.566611 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:18.566573 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" event={"ID":"0fa9c659-33bc-4916-b33f-c1865ee56fe3","Type":"ContainerStarted","Data":"5fad2ecab2f3a140aa82b863aebe7bfa8782d675bdeaca7dba896a1757fb9b2e"} Jul 10 00:26:18.664889 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:18.664851 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:20.580635 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.580597 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" event={"ID":"0721fc09-8f0a-4f00-82e9-9b5fd08171dc","Type":"ContainerStarted","Data":"fc45e056834aa4209494ac50713f528500b57e4429caed46d12a5b230f2296a2"} Jul 10 00:26:20.582700 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.582602 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" event={"ID":"8dfe28e5-4518-4289-a6a9-f4898c92c92f","Type":"ContainerStarted","Data":"4d3e3564e609a82271f79d2c6032e3c39107f857a77d32281901d221d2a96c55"} Jul 10 00:26:20.583104 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.583082 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:20.584624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.584596 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" event={"ID":"c2eaf9cc-d308-43a0-8cca-35f16e605aeb","Type":"ContainerStarted","Data":"90cd28bfbc401c062e8dae70726fe084bc064b687780631ab198aed1d1742a24"} Jul 10 00:26:20.584983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.584922 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:26:20.598568 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.598520 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77865f7b8-lz6q4" podStartSLOduration=1.938801884 podStartE2EDuration="5.598504052s" totalImagesPullingTime="3.659702168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:20.597061048 +0000 UTC m=+552.623539752" watchObservedRunningTime="2026-07-10 00:26:20.598504052 +0000 UTC m=+552.624982757" Jul 10 00:26:20.614502 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.614454 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" podStartSLOduration=2.286400551 podStartE2EDuration="6.614434626s" totalImagesPullingTime="4.328034075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:20.612559773 +0000 UTC m=+552.639038499" watchObservedRunningTime="2026-07-10 00:26:20.614434626 +0000 UTC m=+552.640913336" Jul 10 00:26:20.632323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:20.632270 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" podStartSLOduration=2.321037908 podStartE2EDuration="6.632247959s" totalImagesPullingTime="4.311210051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:20.630198143 +0000 UTC m=+552.656676848" watchObservedRunningTime="2026-07-10 00:26:20.632247959 +0000 UTC m=+552.658726663" Jul 10 00:26:21.521623 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:21.521587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-69dn5" Jul 10 00:26:21.523254 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:21.523232 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qjx67" Jul 10 00:26:22.012524 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.012475 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:26:22.012808 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.012560 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:26:22.598792 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.598760 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" event={"ID":"0fa9c659-33bc-4916-b33f-c1865ee56fe3","Type":"ContainerStarted","Data":"b0ef83a396ce451d6d1c998959add590a7956da81601461486d82347ef0ff445"} Jul 10 00:26:22.598985 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.598819 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:22.620418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.620356 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" podStartSLOduration=1.718833814 podStartE2EDuration="5.620337376s" totalImagesPullingTime="3.901503562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:22.618006501 +0000 UTC m=+554.644485210" watchObservedRunningTime="2026-07-10 00:26:22.620337376 +0000 UTC m=+554.646816083" Jul 10 00:26:22.795761 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.795722 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:22.893820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:22.893720 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:23.336608 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:23.336573 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:23.437006 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:23.436973 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:23.606359 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:23.606276 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-4l2fh" Jul 10 00:26:23.643716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:23.643684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:26.427707 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:26.427670 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:26:51.593873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:51.593844 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zl9h7" Jul 10 00:26:51.596730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:26:51.596715 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzbxk" Jul 10 00:27:06.916786 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:06.916745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:08.119317 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:08.119262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:08.456463 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:08.456380 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:27:08.457377 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:08.457355 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:27:22.671882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.671846 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj"] Jul 10 00:27:22.675541 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.675524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.677749 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.677726 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lc6p2\"" Jul 10 00:27:22.677882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.677751 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 00:27:22.678560 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.678545 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 00:27:22.682554 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.682531 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj"] Jul 10 00:27:22.820303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.820269 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.820303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.820306 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.820549 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.820351 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcqj\" (UniqueName: \"kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.921175 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.921143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.921328 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.921178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.921328 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.921225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcqj\" (UniqueName: \"kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.921551 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.921532 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.921587 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.921560 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.929385 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.929321 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcqj\" (UniqueName: \"kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:22.986251 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:22.986225 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:23.073688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.073657 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4"] Jul 10 00:27:23.077750 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.077452 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.085850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.085823 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4"] Jul 10 00:27:23.132465 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.132441 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj"] Jul 10 00:27:23.133771 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:23.133751 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f8a1f1_445c_4c1a_86d0_ed3add531597.slice/crio-e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0 WatchSource:0}: Error finding container e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0: Status 404 returned error can't find the container with id e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0 Jul 10 00:27:23.223761 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.223722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.223761 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.223763 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.223984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.223809 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p578\" (UniqueName: \"kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.324859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.324829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.325075 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.324865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.325075 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.324909 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p578\" (UniqueName: \"kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.325266 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.325245 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.325322 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.325277 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.333375 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.333350 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p578\" (UniqueName: \"kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.391099 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.391073 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:23.471787 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.471762 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7"] Jul 10 00:27:23.475602 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.475583 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.483587 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.483517 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7"] Jul 10 00:27:23.526391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.526370 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4"] Jul 10 00:27:23.527851 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:23.527820 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f6a9f1_37c1_444f_9839_051b3d89f3af.slice/crio-802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b WatchSource:0}: Error finding container 802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b: Status 404 returned error can't find the container with id 802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b Jul 10 00:27:23.627496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.627471 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.627625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.627529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bx4k\" (UniqueName: \"kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.627677 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.627629 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.728856 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.728783 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.728856 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.728830 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.729261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.728882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bx4k\" (UniqueName: \"kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.729309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.729294 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.729369 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.729351 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.736995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.736978 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bx4k\" (UniqueName: \"kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.788971 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.788929 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:23.875479 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.875449 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp"] Jul 10 00:27:23.879233 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.879213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:23.885730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.885710 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp"] Jul 10 00:27:23.905607 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.905569 2572 generic.go:356] "Generic (PLEG): container finished" podID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerID="d4605ad308a4681e6593b90c9bcd48d0e7191fe9df410bff4678f969ca5c8c7a" exitCode=0 Jul 10 00:27:23.905766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.905656 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerDied","Data":"d4605ad308a4681e6593b90c9bcd48d0e7191fe9df410bff4678f969ca5c8c7a"} Jul 10 00:27:23.905766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.905736 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerStarted","Data":"e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0"} Jul 10 00:27:23.906375 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.906355 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:27:23.907489 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.907465 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerID="b57acfa5fa57f4d7ca45e6fed780c555b66605186d72cb0ad1a4e6e21305ed42" exitCode=0 Jul 10 00:27:23.907581 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.907562 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" event={"ID":"b3f6a9f1-37c1-444f-9839-051b3d89f3af","Type":"ContainerDied","Data":"b57acfa5fa57f4d7ca45e6fed780c555b66605186d72cb0ad1a4e6e21305ed42"} Jul 10 00:27:23.907649 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.907601 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" event={"ID":"b3f6a9f1-37c1-444f-9839-051b3d89f3af","Type":"ContainerStarted","Data":"802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b"} Jul 10 00:27:23.922580 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:23.922541 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7"] Jul 10 00:27:23.924758 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:23.924733 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb454caa_5774_40bb_899c_7f9979f36f5d.slice/crio-e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777 WatchSource:0}: Error finding container e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777: Status 404 returned error can't find the container with id e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777 Jul 10 00:27:24.031377 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.031333 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.031574 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.031541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.031729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.031589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhmlk\" (UniqueName: \"kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.132575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.132542 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.132728 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.132581 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhmlk\" (UniqueName: \"kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.132728 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.132607 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.132932 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.132908 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.132996 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.132927 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.141256 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.141230 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhmlk\" (UniqueName: \"kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.191828 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.191792 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:24.322515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.322486 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp"] Jul 10 00:27:24.323845 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:24.323820 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c94478_edd5_472e_8d74_7121197d8f75.slice/crio-eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7 WatchSource:0}: Error finding container eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7: Status 404 returned error can't find the container with id eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7 Jul 10 00:27:24.914360 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.914336 2572 generic.go:356] "Generic (PLEG): container finished" podID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerID="fb1420890a43978d2b72e60f3e71fba813c25fdc6597206081542778963ab5c1" exitCode=0 Jul 10 00:27:24.914691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.914422 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" event={"ID":"eb454caa-5774-40bb-899c-7f9979f36f5d","Type":"ContainerDied","Data":"fb1420890a43978d2b72e60f3e71fba813c25fdc6597206081542778963ab5c1"} Jul 10 00:27:24.914691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.914475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" event={"ID":"eb454caa-5774-40bb-899c-7f9979f36f5d","Type":"ContainerStarted","Data":"e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777"} Jul 10 00:27:24.916154 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.916129 2572 generic.go:356] "Generic (PLEG): container finished" podID="37c94478-edd5-472e-8d74-7121197d8f75" containerID="58365006f53ef5131c32ee301519136ed94044ec1d4a4eab3ba623b8061c8d78" exitCode=0 Jul 10 00:27:24.916257 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.916213 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" event={"ID":"37c94478-edd5-472e-8d74-7121197d8f75","Type":"ContainerDied","Data":"58365006f53ef5131c32ee301519136ed94044ec1d4a4eab3ba623b8061c8d78"} Jul 10 00:27:24.916320 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.916268 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" event={"ID":"37c94478-edd5-472e-8d74-7121197d8f75","Type":"ContainerStarted","Data":"eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7"} Jul 10 00:27:24.918327 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:24.918230 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerStarted","Data":"09fe4dbea35f0658a3f8058d7dbbce6e711f4c9e8225a904206ab9cc9169a844"} Jul 10 00:27:25.925477 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.925450 2572 generic.go:356] "Generic (PLEG): container finished" podID="37c94478-edd5-472e-8d74-7121197d8f75" containerID="a87d8925a0e275e5b297a6037cdd6852286c7eaefdd9645431bd3f0454e26ff6" exitCode=0 Jul 10 00:27:25.925873 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.925539 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" event={"ID":"37c94478-edd5-472e-8d74-7121197d8f75","Type":"ContainerDied","Data":"a87d8925a0e275e5b297a6037cdd6852286c7eaefdd9645431bd3f0454e26ff6"} Jul 10 00:27:25.927419 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.927397 2572 generic.go:356] "Generic (PLEG): container finished" podID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerID="09fe4dbea35f0658a3f8058d7dbbce6e711f4c9e8225a904206ab9cc9169a844" exitCode=0 Jul 10 00:27:25.927510 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.927483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerDied","Data":"09fe4dbea35f0658a3f8058d7dbbce6e711f4c9e8225a904206ab9cc9169a844"} Jul 10 00:27:25.929456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.929436 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerID="654f809813bc2fff61b0844a2f3c4c6e1bbf0bba865f0015ce88d1ae3608014f" exitCode=0 Jul 10 00:27:25.929545 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.929482 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" event={"ID":"b3f6a9f1-37c1-444f-9839-051b3d89f3af","Type":"ContainerDied","Data":"654f809813bc2fff61b0844a2f3c4c6e1bbf0bba865f0015ce88d1ae3608014f"} Jul 10 00:27:25.931433 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.931416 2572 generic.go:356] "Generic (PLEG): container finished" podID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerID="c9fa58476aea0c608c18ceb5d7606c15c4b0abeb6a7be1e52a2afda8d9523660" exitCode=0 Jul 10 00:27:25.931527 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:25.931461 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" event={"ID":"eb454caa-5774-40bb-899c-7f9979f36f5d","Type":"ContainerDied","Data":"c9fa58476aea0c608c18ceb5d7606c15c4b0abeb6a7be1e52a2afda8d9523660"} Jul 10 00:27:26.938745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.938709 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerID="0d362a50ce9a52d384fdaf58e41107f7065f2fb11b48a631497843e3484adb83" exitCode=0 Jul 10 00:27:26.939222 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.938781 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" event={"ID":"b3f6a9f1-37c1-444f-9839-051b3d89f3af","Type":"ContainerDied","Data":"0d362a50ce9a52d384fdaf58e41107f7065f2fb11b48a631497843e3484adb83"} Jul 10 00:27:26.940930 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.940905 2572 generic.go:356] "Generic (PLEG): container finished" podID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerID="15d59995ed7c69e53898b7f188277d3a69185b7ff682d3daee5c294f8c388e0e" exitCode=0 Jul 10 00:27:26.941048 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.940988 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" event={"ID":"eb454caa-5774-40bb-899c-7f9979f36f5d","Type":"ContainerDied","Data":"15d59995ed7c69e53898b7f188277d3a69185b7ff682d3daee5c294f8c388e0e"} Jul 10 00:27:26.943105 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.943086 2572 generic.go:356] "Generic (PLEG): container finished" podID="37c94478-edd5-472e-8d74-7121197d8f75" containerID="8c98508aa6e39bdb57b89b545609547513ad5a330cdf00ab5ad5555f4a39db08" exitCode=0 Jul 10 00:27:26.943194 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.943164 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" event={"ID":"37c94478-edd5-472e-8d74-7121197d8f75","Type":"ContainerDied","Data":"8c98508aa6e39bdb57b89b545609547513ad5a330cdf00ab5ad5555f4a39db08"} Jul 10 00:27:26.945186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.945167 2572 generic.go:356] "Generic (PLEG): container finished" podID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerID="03446c3015454fe0f856386c854777027d84e78fb50f95bd5d3447ffd6b4aaff" exitCode=0 Jul 10 00:27:26.945260 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:26.945203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerDied","Data":"03446c3015454fe0f856386c854777027d84e78fb50f95bd5d3447ffd6b4aaff"} Jul 10 00:27:28.097854 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.097823 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:28.157801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.157777 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:28.161329 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.161309 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:28.164724 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.164705 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:28.270576 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270545 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util\") pod \"37c94478-edd5-472e-8d74-7121197d8f75\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270604 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxcqj\" (UniqueName: \"kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj\") pod \"98f8a1f1-445c-4c1a-86d0-ed3add531597\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270624 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util\") pod \"98f8a1f1-445c-4c1a-86d0-ed3add531597\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270640 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle\") pod \"98f8a1f1-445c-4c1a-86d0-ed3add531597\" (UID: \"98f8a1f1-445c-4c1a-86d0-ed3add531597\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270655 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bx4k\" (UniqueName: \"kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k\") pod \"eb454caa-5774-40bb-899c-7f9979f36f5d\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270692 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util\") pod \"eb454caa-5774-40bb-899c-7f9979f36f5d\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " Jul 10 00:27:28.270756 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270718 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle\") pod \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " Jul 10 00:27:28.271124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270761 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util\") pod \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " Jul 10 00:27:28.271124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270786 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle\") pod \"eb454caa-5774-40bb-899c-7f9979f36f5d\" (UID: \"eb454caa-5774-40bb-899c-7f9979f36f5d\") " Jul 10 00:27:28.271124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270823 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhmlk\" (UniqueName: \"kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk\") pod \"37c94478-edd5-472e-8d74-7121197d8f75\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " Jul 10 00:27:28.271124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270858 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle\") pod \"37c94478-edd5-472e-8d74-7121197d8f75\" (UID: \"37c94478-edd5-472e-8d74-7121197d8f75\") " Jul 10 00:27:28.271124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.270884 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p578\" (UniqueName: \"kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578\") pod \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\" (UID: \"b3f6a9f1-37c1-444f-9839-051b3d89f3af\") " Jul 10 00:27:28.271544 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.271272 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle" (OuterVolumeSpecName: "bundle") pod "98f8a1f1-445c-4c1a-86d0-ed3add531597" (UID: "98f8a1f1-445c-4c1a-86d0-ed3add531597"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.273424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.272022 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle" (OuterVolumeSpecName: "bundle") pod "eb454caa-5774-40bb-899c-7f9979f36f5d" (UID: "eb454caa-5774-40bb-899c-7f9979f36f5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.273424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.272099 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle" (OuterVolumeSpecName: "bundle") pod "b3f6a9f1-37c1-444f-9839-051b3d89f3af" (UID: "b3f6a9f1-37c1-444f-9839-051b3d89f3af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.273934 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.273809 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k" (OuterVolumeSpecName: "kube-api-access-2bx4k") pod "eb454caa-5774-40bb-899c-7f9979f36f5d" (UID: "eb454caa-5774-40bb-899c-7f9979f36f5d"). InnerVolumeSpecName "kube-api-access-2bx4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:27:28.274134 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.274083 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578" (OuterVolumeSpecName: "kube-api-access-7p578") pod "b3f6a9f1-37c1-444f-9839-051b3d89f3af" (UID: "b3f6a9f1-37c1-444f-9839-051b3d89f3af"). InnerVolumeSpecName "kube-api-access-7p578". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:27:28.274134 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.274092 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj" (OuterVolumeSpecName: "kube-api-access-wxcqj") pod "98f8a1f1-445c-4c1a-86d0-ed3add531597" (UID: "98f8a1f1-445c-4c1a-86d0-ed3add531597"). InnerVolumeSpecName "kube-api-access-wxcqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:27:28.274502 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.274476 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle" (OuterVolumeSpecName: "bundle") pod "37c94478-edd5-472e-8d74-7121197d8f75" (UID: "37c94478-edd5-472e-8d74-7121197d8f75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.274824 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.274802 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk" (OuterVolumeSpecName: "kube-api-access-qhmlk") pod "37c94478-edd5-472e-8d74-7121197d8f75" (UID: "37c94478-edd5-472e-8d74-7121197d8f75"). InnerVolumeSpecName "kube-api-access-qhmlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:27:28.278832 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.278808 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util" (OuterVolumeSpecName: "util") pod "37c94478-edd5-472e-8d74-7121197d8f75" (UID: "37c94478-edd5-472e-8d74-7121197d8f75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.280138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.280119 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util" (OuterVolumeSpecName: "util") pod "b3f6a9f1-37c1-444f-9839-051b3d89f3af" (UID: "b3f6a9f1-37c1-444f-9839-051b3d89f3af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.280300 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.280278 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util" (OuterVolumeSpecName: "util") pod "98f8a1f1-445c-4c1a-86d0-ed3add531597" (UID: "98f8a1f1-445c-4c1a-86d0-ed3add531597"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.280714 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.280697 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util" (OuterVolumeSpecName: "util") pod "eb454caa-5774-40bb-899c-7f9979f36f5d" (UID: "eb454caa-5774-40bb-899c-7f9979f36f5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:27:28.372412 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372381 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxcqj\" (UniqueName: \"kubernetes.io/projected/98f8a1f1-445c-4c1a-86d0-ed3add531597-kube-api-access-wxcqj\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372412 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372405 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372412 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372414 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f8a1f1-445c-4c1a-86d0-ed3add531597-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372423 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bx4k\" (UniqueName: \"kubernetes.io/projected/eb454caa-5774-40bb-899c-7f9979f36f5d-kube-api-access-2bx4k\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372432 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372440 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372447 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3f6a9f1-37c1-444f-9839-051b3d89f3af-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372454 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb454caa-5774-40bb-899c-7f9979f36f5d-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372461 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhmlk\" (UniqueName: \"kubernetes.io/projected/37c94478-edd5-472e-8d74-7121197d8f75-kube-api-access-qhmlk\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372469 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372477 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p578\" (UniqueName: \"kubernetes.io/projected/b3f6a9f1-37c1-444f-9839-051b3d89f3af-kube-api-access-7p578\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.372621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.372484 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37c94478-edd5-472e-8d74-7121197d8f75-util\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:27:28.957880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.957791 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" Jul 10 00:27:28.957880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.957817 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp" event={"ID":"37c94478-edd5-472e-8d74-7121197d8f75","Type":"ContainerDied","Data":"eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7"} Jul 10 00:27:28.957880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.957873 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eee10b6026c92aa1eaa8725b51d29032a514b4259028df645374fc2339e4b5e7" Jul 10 00:27:28.959818 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.959795 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" event={"ID":"98f8a1f1-445c-4c1a-86d0-ed3add531597","Type":"ContainerDied","Data":"e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0"} Jul 10 00:27:28.959972 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.959821 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj" Jul 10 00:27:28.959972 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.959835 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5647570937d428fc0b1459f3b0e0262dde9edb43924e99965c018c9141797f0" Jul 10 00:27:28.961826 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.961803 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" event={"ID":"b3f6a9f1-37c1-444f-9839-051b3d89f3af","Type":"ContainerDied","Data":"802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b"} Jul 10 00:27:28.961999 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.961980 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="802ec8c56cec4b4f5599c4a3e27cad45cc45b3c7711493d9dc1cd4cc210d4c1b" Jul 10 00:27:28.961999 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.961827 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4" Jul 10 00:27:28.963814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.963764 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" event={"ID":"eb454caa-5774-40bb-899c-7f9979f36f5d","Type":"ContainerDied","Data":"e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777"} Jul 10 00:27:28.963814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.963800 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e66d00603530b34c335da1c0fef400172f9523d7aa9b4f50d532941406d49777" Jul 10 00:27:28.964010 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:28.963859 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7" Jul 10 00:27:32.374646 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:32.374572 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:32.427943 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:32.427910 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:34.325225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:34.325192 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:34.392407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:34.392361 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:35.358706 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:35.358679 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:39.205854 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:39.205823 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:40.462321 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462292 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c8f4c8746-bcszv"] Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462657 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="pull" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462668 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="pull" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462678 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462684 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462690 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462695 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462701 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462706 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="extract" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462716 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="util" Jul 10 00:27:40.462721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462721 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462731 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462740 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462752 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462761 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462786 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462795 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462802 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462807 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462814 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="extract" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462819 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="extract" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462827 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462832 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="util" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462838 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462843 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="pull" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462917 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37c94478-edd5-472e-8d74-7121197d8f75" containerName="extract" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462932 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb454caa-5774-40bb-899c-7f9979f36f5d" containerName="extract" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462941 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f6a9f1-37c1-444f-9839-051b3d89f3af" containerName="extract" Jul 10 00:27:40.463077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.462967 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f8a1f1-445c-4c1a-86d0-ed3add531597" containerName="extract" Jul 10 00:27:40.466065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.466048 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.478886 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.478866 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8f4c8746-bcszv"] Jul 10 00:27:40.479225 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-console-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479248 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-service-ca\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479273 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzkbm\" (UniqueName: \"kubernetes.io/projected/470aea4f-ea26-4452-9183-2f7115ce0355-kube-api-access-xzkbm\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479309 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479307 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-oauth-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479528 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479327 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-oauth-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.479528 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.479400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-trusted-ca-bundle\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.525496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.525465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:40.580764 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.580735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-trusted-ca-bundle\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.580920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.580773 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-console-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.580920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.580800 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-service-ca\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.580920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.580825 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzkbm\" (UniqueName: \"kubernetes.io/projected/470aea4f-ea26-4452-9183-2f7115ce0355-kube-api-access-xzkbm\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-oauth-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581234 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-oauth-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581528 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581497 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-console-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581693 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-trusted-ca-bundle\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581751 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-oauth-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.581819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.581775 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/470aea4f-ea26-4452-9183-2f7115ce0355-service-ca\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.583340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.583318 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-serving-cert\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.583496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.583479 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/470aea4f-ea26-4452-9183-2f7115ce0355-console-oauth-config\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.589630 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.589608 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzkbm\" (UniqueName: \"kubernetes.io/projected/470aea4f-ea26-4452-9183-2f7115ce0355-kube-api-access-xzkbm\") pod \"console-5c8f4c8746-bcszv\" (UID: \"470aea4f-ea26-4452-9183-2f7115ce0355\") " pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.625065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.625043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:40.717136 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.717042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:40.775148 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.775116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:40.909927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.909897 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c8f4c8746-bcszv"] Jul 10 00:27:40.911673 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:40.911649 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470aea4f_ea26_4452_9183_2f7115ce0355.slice/crio-b012ee000bb00b054965a27fbd76ec2d8db6a9d59132bc36ab68b4c728825a18 WatchSource:0}: Error finding container b012ee000bb00b054965a27fbd76ec2d8db6a9d59132bc36ab68b4c728825a18: Status 404 returned error can't find the container with id b012ee000bb00b054965a27fbd76ec2d8db6a9d59132bc36ab68b4c728825a18 Jul 10 00:27:40.961663 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:40.961632 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:41.028842 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.028799 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8f4c8746-bcszv" event={"ID":"470aea4f-ea26-4452-9183-2f7115ce0355","Type":"ContainerStarted","Data":"03b0a671034b6c8f6ee4d507801588a2a80fd93862c3d74877d63891201a6c09"} Jul 10 00:27:41.029036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.028863 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c8f4c8746-bcszv" event={"ID":"470aea4f-ea26-4452-9183-2f7115ce0355","Type":"ContainerStarted","Data":"b012ee000bb00b054965a27fbd76ec2d8db6a9d59132bc36ab68b4c728825a18"} Jul 10 00:27:41.052034 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.051990 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c8f4c8746-bcszv" podStartSLOduration=1.051974257 podStartE2EDuration="1.051974257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:41.049308653 +0000 UTC m=+633.075787358" watchObservedRunningTime="2026-07-10 00:27:41.051974257 +0000 UTC m=+633.078452951" Jul 10 00:27:41.255536 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.255504 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:41.357567 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.357535 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:41.999500 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:41.999465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:42.889069 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.889030 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j"] Jul 10 00:27:42.893428 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.893395 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:42.898373 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.898345 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 00:27:42.898515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.898424 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c9ssx\"" Jul 10 00:27:42.898578 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.898563 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 00:27:42.906833 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:42.906805 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j"] Jul 10 00:27:43.008667 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.008630 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4zjm\" (UniqueName: \"kubernetes.io/projected/b223879f-13ef-4166-b16c-4625b374efdd-kube-api-access-c4zjm\") pod \"authorino-operator-75cc8d5cb7-n2k5j\" (UID: \"b223879f-13ef-4166-b16c-4625b374efdd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:43.109497 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.109461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4zjm\" (UniqueName: \"kubernetes.io/projected/b223879f-13ef-4166-b16c-4625b374efdd-kube-api-access-c4zjm\") pod \"authorino-operator-75cc8d5cb7-n2k5j\" (UID: \"b223879f-13ef-4166-b16c-4625b374efdd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:43.118671 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.118631 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4zjm\" (UniqueName: \"kubernetes.io/projected/b223879f-13ef-4166-b16c-4625b374efdd-kube-api-access-c4zjm\") pod \"authorino-operator-75cc8d5cb7-n2k5j\" (UID: \"b223879f-13ef-4166-b16c-4625b374efdd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:43.214665 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.214572 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:43.349875 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.349848 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j"] Jul 10 00:27:43.350903 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:27:43.350880 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb223879f_13ef_4166_b16c_4625b374efdd.slice/crio-285be2aeef6b2c8e9030870fd3adbce180026c1729ced5668852043a5ad55d97 WatchSource:0}: Error finding container 285be2aeef6b2c8e9030870fd3adbce180026c1729ced5668852043a5ad55d97: Status 404 returned error can't find the container with id 285be2aeef6b2c8e9030870fd3adbce180026c1729ced5668852043a5ad55d97 Jul 10 00:27:43.601911 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.601884 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:43.694547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:43.694511 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:44.049298 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:44.049259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" event={"ID":"b223879f-13ef-4166-b16c-4625b374efdd","Type":"ContainerStarted","Data":"285be2aeef6b2c8e9030870fd3adbce180026c1729ced5668852043a5ad55d97"} Jul 10 00:27:45.771907 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:45.771868 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:46.070518 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:46.070480 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" event={"ID":"b223879f-13ef-4166-b16c-4625b374efdd","Type":"ContainerStarted","Data":"8fec4333e73417eb756837b50a7842669ec86fdae2c1678e7f9ba50d61b07b77"} Jul 10 00:27:46.070677 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:46.070580 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:46.091133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:46.091079 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" podStartSLOduration=1.9109933620000001 podStartE2EDuration="4.091064468s" totalImagesPullingTime="2.180071106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:46.088677152 +0000 UTC m=+638.115155862" watchObservedRunningTime="2026-07-10 00:27:46.091064468 +0000 UTC m=+638.117543165" Jul 10 00:27:47.601255 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:47.601221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:49.309527 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:49.309490 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:49.406146 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:49.406112 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:49.506368 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:49.506338 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:49.991743 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:49.991717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:50.775409 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:50.775377 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:50.775813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:50.775432 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:50.780005 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:50.779984 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:51.100926 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:51.100836 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c8f4c8746-bcszv" Jul 10 00:27:51.146853 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:51.146816 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:27:51.244172 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:51.244136 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:51.342244 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:51.342213 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:51.393449 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:51.393368 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:57.078263 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:57.078233 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n2k5j" Jul 10 00:27:59.589660 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.589624 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:59.690483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.690446 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:27:59.898261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.898190 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb"] Jul 10 00:27:59.903620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.903600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:27:59.906875 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.906857 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jdxtx\"" Jul 10 00:27:59.931254 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.931228 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb"] Jul 10 00:27:59.971138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:27:59.971113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6vq\" (UniqueName: \"kubernetes.io/projected/8c5692a8-275f-458c-ad74-3ab741c03dce-kube-api-access-nd6vq\") pod \"limitador-operator-controller-manager-859d7b78bd-wdxxb\" (UID: \"8c5692a8-275f-458c-ad74-3ab741c03dce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:00.072420 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:00.072366 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6vq\" (UniqueName: \"kubernetes.io/projected/8c5692a8-275f-458c-ad74-3ab741c03dce-kube-api-access-nd6vq\") pod \"limitador-operator-controller-manager-859d7b78bd-wdxxb\" (UID: \"8c5692a8-275f-458c-ad74-3ab741c03dce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:00.081645 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:00.081615 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6vq\" (UniqueName: \"kubernetes.io/projected/8c5692a8-275f-458c-ad74-3ab741c03dce-kube-api-access-nd6vq\") pod \"limitador-operator-controller-manager-859d7b78bd-wdxxb\" (UID: \"8c5692a8-275f-458c-ad74-3ab741c03dce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:00.214263 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:00.214182 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:00.349645 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:00.349619 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb"] Jul 10 00:28:00.351773 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:28:00.351746 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5692a8_275f_458c_ad74_3ab741c03dce.slice/crio-46cac9e5d85a5878cc13ce950ce39c0c8dddf3ddc211b31344dec771e3480168 WatchSource:0}: Error finding container 46cac9e5d85a5878cc13ce950ce39c0c8dddf3ddc211b31344dec771e3480168: Status 404 returned error can't find the container with id 46cac9e5d85a5878cc13ce950ce39c0c8dddf3ddc211b31344dec771e3480168 Jul 10 00:28:01.152295 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:01.152256 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" event={"ID":"8c5692a8-275f-458c-ad74-3ab741c03dce","Type":"ContainerStarted","Data":"46cac9e5d85a5878cc13ce950ce39c0c8dddf3ddc211b31344dec771e3480168"} Jul 10 00:28:02.041297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:02.041262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:02.141416 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:02.141382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:03.165749 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:03.165714 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" event={"ID":"8c5692a8-275f-458c-ad74-3ab741c03dce","Type":"ContainerStarted","Data":"99a5ea0df92aa5745e2489bf4589cadf0b87a6318e73c9049a1b71e8700254ad"} Jul 10 00:28:03.166138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:03.165775 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:03.182702 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:03.182655 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" podStartSLOduration=2.265920335 podStartE2EDuration="4.182639008s" totalImagesPullingTime="1.916718673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:28:03.181276751 +0000 UTC m=+655.207755468" watchObservedRunningTime="2026-07-10 00:28:03.182639008 +0000 UTC m=+655.209117712" Jul 10 00:28:06.777575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:06.777540 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:06.880346 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:06.880317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:10.589632 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:10.589600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:10.685230 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:10.685201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:14.174361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:14.174327 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wdxxb" Jul 10 00:28:16.175232 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.175191 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b6d7d697f-bp7bs" podUID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" containerName="console" containerID="cri-o://31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3" gracePeriod=15 Jul 10 00:28:16.424498 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.424477 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:28:16.519646 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519622 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.519813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519655 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.519813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519681 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scxw7\" (UniqueName: \"kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.519813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519710 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.519813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519745 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.519813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519800 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.520091 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.519940 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config\") pod \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\" (UID: \"0b9738a3-d632-4225-a99e-d0e77e8b1b00\") " Jul 10 00:28:16.520152 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520124 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:28:16.520214 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520190 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config" (OuterVolumeSpecName: "console-config") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:28:16.520214 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520198 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:28:16.520326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520287 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:28:16.520326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520300 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-trusted-ca-bundle\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.520326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520315 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-service-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.520326 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.520325 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.522001 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.521974 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7" (OuterVolumeSpecName: "kube-api-access-scxw7") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "kube-api-access-scxw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:28:16.522135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.522002 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:28:16.522135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.522104 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0b9738a3-d632-4225-a99e-d0e77e8b1b00" (UID: "0b9738a3-d632-4225-a99e-d0e77e8b1b00"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:28:16.620881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.620843 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b9738a3-d632-4225-a99e-d0e77e8b1b00-oauth-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.620881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.620877 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scxw7\" (UniqueName: \"kubernetes.io/projected/0b9738a3-d632-4225-a99e-d0e77e8b1b00-kube-api-access-scxw7\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.620881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.620886 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-serving-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:16.620881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:16.620895 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b9738a3-d632-4225-a99e-d0e77e8b1b00-console-oauth-config\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:28:17.242978 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.242926 2572 generic.go:356] "Generic (PLEG): container finished" podID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" containerID="31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3" exitCode=0 Jul 10 00:28:17.243390 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.242986 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b6d7d697f-bp7bs" Jul 10 00:28:17.243390 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.242995 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6d7d697f-bp7bs" event={"ID":"0b9738a3-d632-4225-a99e-d0e77e8b1b00","Type":"ContainerDied","Data":"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3"} Jul 10 00:28:17.243390 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.243054 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b6d7d697f-bp7bs" event={"ID":"0b9738a3-d632-4225-a99e-d0e77e8b1b00","Type":"ContainerDied","Data":"1ba95bd606e09b0651c0fec72f80aec10573bcc85c70f5881be35b1a355bffcf"} Jul 10 00:28:17.243390 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.243076 2572 scope.go:117] "RemoveContainer" containerID="31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3" Jul 10 00:28:17.253712 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.253693 2572 scope.go:117] "RemoveContainer" containerID="31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3" Jul 10 00:28:17.254032 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:28:17.254010 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3\": container with ID starting with 31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3 not found: ID does not exist" containerID="31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3" Jul 10 00:28:17.254136 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.254044 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3"} err="failed to get container status \"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3\": rpc error: code = NotFound desc = could not find container \"31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3\": container with ID starting with 31c449c48e66abcdeeb25d001946343f16197bda5690583d63cf613d732becd3 not found: ID does not exist" Jul 10 00:28:17.264294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.264270 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:28:17.268093 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:17.268071 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b6d7d697f-bp7bs"] Jul 10 00:28:18.514164 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:18.514121 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" path="/var/lib/kubelet/pods/0b9738a3-d632-4225-a99e-d0e77e8b1b00/volumes" Jul 10 00:28:19.226179 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:19.226145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:19.326433 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:19.326396 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:21.668046 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:21.668006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:21.771995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:21.771941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:26.421725 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:26.421685 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:26.522197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:26.522166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:28.438706 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:28.438674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:28.762462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:28.762417 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:30.433449 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:30.433383 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:30.506542 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:30.506502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:31.594495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:31.594452 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:40.825155 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:40.825095 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:40.879806 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:40.879776 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:41.221601 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:41.221525 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:41.336030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:41.335994 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:44.628181 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:44.628150 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:44.731356 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:44.731319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:44.782638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:44.782600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:45.040043 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:45.040009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:45.124524 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:45.124488 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:45.474292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:45.474217 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:45.538197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:45.538169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:47.847022 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:47.846992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:28:47.947258 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:28:47.947226 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:07.061662 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:07.061564 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:07.159580 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:07.159544 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:07.602670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:07.602637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:19.537995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:19.537941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:19.698374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:19.698333 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:26.753357 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:26.753321 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:26.852109 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:26.852075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:26.947296 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:26.947255 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:27.002432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:27.002396 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:29.684999 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:29.684928 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:29.785251 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:29.785211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:29:38.948636 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:29:38.948592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:00.229921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:00.229887 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:00.671098 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:00.671067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:00.977730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:00.977633 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:01.139170 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:01.139139 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:01.216850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:01.216814 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:11.661305 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:11.661270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:11.747429 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:11.747393 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:23.065666 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:23.065635 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:25.875332 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.875297 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd"] Jul 10 00:30:25.875688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.875676 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" containerName="console" Jul 10 00:30:25.875688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.875688 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" containerName="console" Jul 10 00:30:25.875759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.875743 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b9738a3-d632-4225-a99e-d0e77e8b1b00" containerName="console" Jul 10 00:30:25.878997 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.878977 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.881149 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.881124 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 00:30:25.881262 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.881207 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 00:30:25.882214 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.882197 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ztfmx\"" Jul 10 00:30:25.882214 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.882208 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 00:30:25.886294 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.886272 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd"] Jul 10 00:30:25.905184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.905153 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:25.957045 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957015 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eeffbb2-a028-48cb-89ae-a72baf787737-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.957219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.957219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957110 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.957219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957202 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q54vl\" (UniqueName: \"kubernetes.io/projected/7eeffbb2-a028-48cb-89ae-a72baf787737-kube-api-access-q54vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.957371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957237 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:25.957371 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:25.957306 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.057876 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.057837 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.057918 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q54vl\" (UniqueName: \"kubernetes.io/projected/7eeffbb2-a028-48cb-89ae-a72baf787737-kube-api-access-q54vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.057983 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058212 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058070 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eeffbb2-a028-48cb-89ae-a72baf787737-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058212 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058115 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058388 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058481 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058426 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.058538 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.058485 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.060337 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.060318 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eeffbb2-a028-48cb-89ae-a72baf787737-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.060577 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.060561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eeffbb2-a028-48cb-89ae-a72baf787737-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.065802 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.065780 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q54vl\" (UniqueName: \"kubernetes.io/projected/7eeffbb2-a028-48cb-89ae-a72baf787737-kube-api-access-q54vl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd\" (UID: \"7eeffbb2-a028-48cb-89ae-a72baf787737\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.191316 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.191224 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:26.332546 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.332520 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd"] Jul 10 00:30:26.333469 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:30:26.333448 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eeffbb2_a028_48cb_89ae_a72baf787737.slice/crio-312184b4f13ec443fa348d87b0b4eb3d0c1350e12e3161396338f67eca762b6c WatchSource:0}: Error finding container 312184b4f13ec443fa348d87b0b4eb3d0c1350e12e3161396338f67eca762b6c: Status 404 returned error can't find the container with id 312184b4f13ec443fa348d87b0b4eb3d0c1350e12e3161396338f67eca762b6c Jul 10 00:30:26.958782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:26.958748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" event={"ID":"7eeffbb2-a028-48cb-89ae-a72baf787737","Type":"ContainerStarted","Data":"312184b4f13ec443fa348d87b0b4eb3d0c1350e12e3161396338f67eca762b6c"} Jul 10 00:30:27.966015 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:27.965980 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" event={"ID":"7eeffbb2-a028-48cb-89ae-a72baf787737","Type":"ContainerStarted","Data":"f8ee049d6ad5fd20c6f6d387405cabe7a77ec595b7ef5b8625ce8267cb894da2"} Jul 10 00:30:27.982743 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:27.982686 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" podStartSLOduration=1.837950172 podStartE2EDuration="2.982666908s" totalImagesPullingTime="1.144716736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:30:27.982444873 +0000 UTC m=+800.008923590" watchObservedRunningTime="2026-07-10 00:30:27.982666908 +0000 UTC m=+800.009145613" Jul 10 00:30:31.676634 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:31.676548 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:36.192115 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:36.192078 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:36.192545 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:36.192133 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:36.204701 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:36.204671 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:37.024975 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:37.024922 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd" Jul 10 00:30:42.500590 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:42.500554 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:30:54.167722 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:30:54.167690 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:31:14.017682 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:31:14.017645 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:31:29.507478 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:31:29.507443 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:31:44.961199 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:31:44.961166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:31:45.016123 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:31:45.016087 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:32:08.491748 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:32:08.491715 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:32:08.494816 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:32:08.494794 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:32:49.125322 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:32:49.125290 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:32:50.925399 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:32:50.925361 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:05.018559 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:05.018524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:06.325001 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:06.324969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:08.888779 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:08.888744 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:08.941210 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:08.941186 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:12.816311 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:12.816279 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:19.826699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:19.826660 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:51.840681 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:51.840642 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:51.937915 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:51.937883 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:52.115320 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:52.115236 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:56.507114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:56.507079 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:57.756317 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:57.756283 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:57.857852 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:57.857815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:57.962844 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:57.962807 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:34:58.192016 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:34:58.191900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:35:03.925169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:35:03.925068 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:35:04.024603 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:35:04.024573 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:36:44.945598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:36:44.945522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:36:45.034211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:36:45.034165 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:37:08.527832 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:37:08.527802 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:37:08.532934 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:37:08.532914 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:40:32.179364 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:40:32.179332 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:42:08.562548 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:42:08.562519 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:42:08.570092 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:42:08.570068 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:43:40.328084 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:43:40.328027 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:45:00.138604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.138571 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:45:00.142108 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.142088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" Jul 10 00:45:00.144242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.144219 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xsl9\"" Jul 10 00:45:00.151491 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.151469 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" Jul 10 00:45:00.156650 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.156629 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:45:00.284977 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.284933 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:45:00.286876 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:45:00.286839 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b621612_d6fc_4aee_a6da_c275c54a2ce2.slice/crio-6ab72e57d9ab4e89f9735eb4eeb9ac381d62a358780930bc4fca15c107ac9fd0 WatchSource:0}: Error finding container 6ab72e57d9ab4e89f9735eb4eeb9ac381d62a358780930bc4fca15c107ac9fd0: Status 404 returned error can't find the container with id 6ab72e57d9ab4e89f9735eb4eeb9ac381d62a358780930bc4fca15c107ac9fd0 Jul 10 00:45:00.288694 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.288679 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:45:00.729752 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:00.729719 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" event={"ID":"8b621612-d6fc-4aee-a6da-c275c54a2ce2","Type":"ContainerStarted","Data":"6ab72e57d9ab4e89f9735eb4eeb9ac381d62a358780930bc4fca15c107ac9fd0"} Jul 10 00:45:03.748016 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:03.747982 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" event={"ID":"8b621612-d6fc-4aee-a6da-c275c54a2ce2","Type":"ContainerStarted","Data":"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0"} Jul 10 00:45:03.764522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:45:03.764463 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" podStartSLOduration=1.264593928 podStartE2EDuration="3.764446267s" totalImagesPullingTime="2.499852339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:45:03.761178755 +0000 UTC m=+1675.787657459" watchObservedRunningTime="2026-07-10 00:45:03.764446267 +0000 UTC m=+1675.790924976" Jul 10 00:46:00.010802 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:00.010763 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:46:00.011377 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:00.010968 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" podUID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" containerName="cleanup" containerID="cri-o://438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0" gracePeriod=30 Jul 10 00:46:07.776248 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:07.776206 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:13.306564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:13.306518 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:21.628364 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:21.628325 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:26.897613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:26.897576 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:30.193591 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.193571 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" Jul 10 00:46:30.238675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.238645 2572 generic.go:356] "Generic (PLEG): container finished" podID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" containerID="438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0" exitCode=137 Jul 10 00:46:30.238819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.238690 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" Jul 10 00:46:30.238819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.238720 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" event={"ID":"8b621612-d6fc-4aee-a6da-c275c54a2ce2","Type":"ContainerDied","Data":"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0"} Jul 10 00:46:30.238819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.238766 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw" event={"ID":"8b621612-d6fc-4aee-a6da-c275c54a2ce2","Type":"ContainerDied","Data":"6ab72e57d9ab4e89f9735eb4eeb9ac381d62a358780930bc4fca15c107ac9fd0"} Jul 10 00:46:30.238819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.238793 2572 scope.go:117] "RemoveContainer" containerID="438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0" Jul 10 00:46:30.248158 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.248139 2572 scope.go:117] "RemoveContainer" containerID="438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0" Jul 10 00:46:30.248442 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:46:30.248422 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0\": container with ID starting with 438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0 not found: ID does not exist" containerID="438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0" Jul 10 00:46:30.248515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.248454 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0"} err="failed to get container status \"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0\": rpc error: code = NotFound desc = could not find container \"438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0\": container with ID starting with 438c6193e915aa19b6d0a9c8913b6820e4cdad6f149715ddd89ac3c8aabf19c0 not found: ID does not exist" Jul 10 00:46:30.258722 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.258696 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:46:30.262198 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.262179 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-hnxdw"] Jul 10 00:46:30.376628 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.376514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:30.479691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.479661 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:30.514490 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.514461 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" path="/var/lib/kubelet/pods/8b621612-d6fc-4aee-a6da-c275c54a2ce2/volumes" Jul 10 00:46:30.910597 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:30.910569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:36.532568 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:36.532530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:36.632633 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:36.632604 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:36.781021 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:36.780981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:36.881745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:36.881662 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:37.852608 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.852573 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:37.934066 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.934032 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:37.934617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.934602 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" containerName="cleanup" Jul 10 00:46:37.934660 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.934622 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" containerName="cleanup" Jul 10 00:46:37.934747 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.934736 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b621612-d6fc-4aee-a6da-c275c54a2ce2" containerName="cleanup" Jul 10 00:46:37.935643 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.935620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:37.939417 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.939399 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:37.941932 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.941906 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d4ecd23a-serving-cert\"" Jul 10 00:46:37.942053 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.941988 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 00:46:37.942122 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.942048 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 00:46:37.943201 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.943156 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wbjng\"" Jul 10 00:46:37.946733 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:37.946714 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:38.004077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.004043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:38.024026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.023899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbnq9\" (UniqueName: \"kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.024198 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.024053 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.024198 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.024104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.105739 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.105645 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:38.124882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.124844 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbnq9\" (UniqueName: \"kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.125066 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.124896 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.125066 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.124982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.126052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.126028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.127995 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.127968 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.133089 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.133065 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbnq9\" (UniqueName: \"kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9\") pod \"maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.251321 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.251279 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:38.398454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.398422 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:38.399224 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.399196 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:38.400293 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:46:38.400270 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3662983c_f48a_4371_b864_8d7a05a3b800.slice/crio-07969e4a0ae5e130e3da37757aceee02ab133c346a8fec36d5145b660479a76b WatchSource:0}: Error finding container 07969e4a0ae5e130e3da37757aceee02ab133c346a8fec36d5145b660479a76b: Status 404 returned error can't find the container with id 07969e4a0ae5e130e3da37757aceee02ab133c346a8fec36d5145b660479a76b Jul 10 00:46:38.496718 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:38.496687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:39.291795 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:39.291750 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" event={"ID":"3662983c-f48a-4371-b864-8d7a05a3b800","Type":"ContainerStarted","Data":"07969e4a0ae5e130e3da37757aceee02ab133c346a8fec36d5145b660479a76b"} Jul 10 00:46:40.181870 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:40.181805 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:41.310711 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:41.310675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" event={"ID":"3662983c-f48a-4371-b864-8d7a05a3b800","Type":"ContainerStarted","Data":"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df"} Jul 10 00:46:41.311221 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:41.310741 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:41.328042 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:41.327999 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" podStartSLOduration=2.457166934 podStartE2EDuration="4.327983005s" totalImagesPullingTime="1.870816071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:46:41.325794804 +0000 UTC m=+1773.352273506" watchObservedRunningTime="2026-07-10 00:46:41.327983005 +0000 UTC m=+1773.354461709" Jul 10 00:46:43.273374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.273345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:43.527599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.527522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:43.630319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.630287 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:43.724856 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.724820 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:43.725105 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.725075 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" podUID="3662983c-f48a-4371-b864-8d7a05a3b800" containerName="maas-api" containerID="cri-o://b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df" gracePeriod=30 Jul 10 00:46:43.748088 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.748060 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:43.984159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:43.984137 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:44.073133 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.073098 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbnq9\" (UniqueName: \"kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9\") pod \"3662983c-f48a-4371-b864-8d7a05a3b800\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " Jul 10 00:46:44.073448 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.073143 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls\") pod \"3662983c-f48a-4371-b864-8d7a05a3b800\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " Jul 10 00:46:44.073448 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.073186 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca\") pod \"3662983c-f48a-4371-b864-8d7a05a3b800\" (UID: \"3662983c-f48a-4371-b864-8d7a05a3b800\") " Jul 10 00:46:44.073670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.073646 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3662983c-f48a-4371-b864-8d7a05a3b800" (UID: "3662983c-f48a-4371-b864-8d7a05a3b800"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:46:44.075534 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.075511 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3662983c-f48a-4371-b864-8d7a05a3b800" (UID: "3662983c-f48a-4371-b864-8d7a05a3b800"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:46:44.075534 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.075510 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9" (OuterVolumeSpecName: "kube-api-access-jbnq9") pod "3662983c-f48a-4371-b864-8d7a05a3b800" (UID: "3662983c-f48a-4371-b864-8d7a05a3b800"). InnerVolumeSpecName "kube-api-access-jbnq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:46:44.173937 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.173856 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbnq9\" (UniqueName: \"kubernetes.io/projected/3662983c-f48a-4371-b864-8d7a05a3b800-kube-api-access-jbnq9\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:44.173937 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.173882 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3662983c-f48a-4371-b864-8d7a05a3b800-maas-api-tls\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:44.173937 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.173891 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3662983c-f48a-4371-b864-8d7a05a3b800-openshift-service-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:44.329904 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.329868 2572 generic.go:356] "Generic (PLEG): container finished" podID="3662983c-f48a-4371-b864-8d7a05a3b800" containerID="b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df" exitCode=0 Jul 10 00:46:44.330383 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.329914 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" Jul 10 00:46:44.330383 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.329984 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" event={"ID":"3662983c-f48a-4371-b864-8d7a05a3b800","Type":"ContainerDied","Data":"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df"} Jul 10 00:46:44.330383 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.330047 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc" event={"ID":"3662983c-f48a-4371-b864-8d7a05a3b800","Type":"ContainerDied","Data":"07969e4a0ae5e130e3da37757aceee02ab133c346a8fec36d5145b660479a76b"} Jul 10 00:46:44.330383 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.330082 2572 scope.go:117] "RemoveContainer" containerID="b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df" Jul 10 00:46:44.339721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.339705 2572 scope.go:117] "RemoveContainer" containerID="b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df" Jul 10 00:46:44.339983 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:46:44.339966 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df\": container with ID starting with b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df not found: ID does not exist" containerID="b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df" Jul 10 00:46:44.340050 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.339989 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df"} err="failed to get container status \"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df\": rpc error: code = NotFound desc = could not find container \"b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df\": container with ID starting with b14ea33e9068e9a43b582bf5b52d301e07c57ea60ea900d376a6572e641ba9df not found: ID does not exist" Jul 10 00:46:44.351784 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.351763 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:44.357287 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.357264 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4ecd23a-845dc67cf9-jgsmc"] Jul 10 00:46:44.513057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:44.513023 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3662983c-f48a-4371-b864-8d7a05a3b800" path="/var/lib/kubelet/pods/3662983c-f48a-4371-b864-8d7a05a3b800/volumes" Jul 10 00:46:45.119816 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:45.119777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:49.488159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:49.488129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:49.588675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:49.588641 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:49.715762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:49.715730 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:49.801005 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:49.800972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:50.830516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.830481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:50.888432 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.888403 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:50.888868 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.888851 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3662983c-f48a-4371-b864-8d7a05a3b800" containerName="maas-api" Jul 10 00:46:50.888980 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.888870 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3662983c-f48a-4371-b864-8d7a05a3b800" containerName="maas-api" Jul 10 00:46:50.889048 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.888983 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3662983c-f48a-4371-b864-8d7a05a3b800" containerName="maas-api" Jul 10 00:46:50.893305 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.893287 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:50.895737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.895716 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a1faa6f3-serving-cert\"" Jul 10 00:46:50.895850 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.895763 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 00:46:50.896547 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.896524 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wbjng\"" Jul 10 00:46:50.896660 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.896535 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 00:46:50.908455 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.908423 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:50.919884 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.919862 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:50.932877 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.932852 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:50.932992 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.932915 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prmxm\" (UniqueName: \"kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:50.933036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.933010 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:50.984340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:50.984314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:51.034272 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.034245 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prmxm\" (UniqueName: \"kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.034443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.034286 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.034443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.034328 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.034443 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:46:51.034423 2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-serving-cert: secret "maas-api-e2e-ait-a1faa6f3-serving-cert" not found Jul 10 00:46:51.034620 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:46:51.034481 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls podName:cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7 nodeName:}" failed. No retries permitted until 2026-07-10 00:46:51.534461375 +0000 UTC m=+1783.560940058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls") pod "maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" (UID: "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7") : secret "maas-api-e2e-ait-a1faa6f3-serving-cert" not found Jul 10 00:46:51.035031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.034986 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.043896 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.043871 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prmxm\" (UniqueName: \"kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.162708 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.162626 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:51.365154 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.365101 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:51.465313 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.465206 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:51.539759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.539727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.542300 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.542282 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") pod \"maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.807381 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.807351 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:51.942837 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:51.942810 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:51.943984 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:46:51.943937 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8cc2f8_a11b_4bc5_8887_d25a04a2c2e7.slice/crio-cd1a4b4d4e13f5237864c28a28eba4c4ffcc09bdf764085d360ba67450642634 WatchSource:0}: Error finding container cd1a4b4d4e13f5237864c28a28eba4c4ffcc09bdf764085d360ba67450642634: Status 404 returned error can't find the container with id cd1a4b4d4e13f5237864c28a28eba4c4ffcc09bdf764085d360ba67450642634 Jul 10 00:46:52.378660 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:52.378625 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" event={"ID":"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7","Type":"ContainerStarted","Data":"cd1a4b4d4e13f5237864c28a28eba4c4ffcc09bdf764085d360ba67450642634"} Jul 10 00:46:53.150688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:53.150636 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:53.385793 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:53.385757 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" event={"ID":"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7","Type":"ContainerStarted","Data":"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56"} Jul 10 00:46:53.385973 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:53.385835 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:53.402624 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:53.402518 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" podStartSLOduration=2.9037099189999998 podStartE2EDuration="3.402503662s" totalImagesPullingTime="498.793743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:46:53.401128219 +0000 UTC m=+1785.427606920" watchObservedRunningTime="2026-07-10 00:46:53.402503662 +0000 UTC m=+1785.428982366" Jul 10 00:46:56.244970 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:56.244919 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:56.760300 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:56.760266 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:56.760510 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:56.760483 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" podUID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" containerName="maas-api" containerID="cri-o://a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56" gracePeriod=30 Jul 10 00:46:56.784296 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:56.784270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:46:57.019759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.019684 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:57.087583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.087552 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prmxm\" (UniqueName: \"kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm\") pod \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " Jul 10 00:46:57.087766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.087604 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") pod \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " Jul 10 00:46:57.087766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.087650 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca\") pod \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\" (UID: \"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7\") " Jul 10 00:46:57.088070 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.088047 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" (UID: "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:46:57.089897 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.089870 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" (UID: "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:46:57.089897 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.089882 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm" (OuterVolumeSpecName: "kube-api-access-prmxm") pod "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" (UID: "cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7"). InnerVolumeSpecName "kube-api-access-prmxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:46:57.188993 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.188935 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prmxm\" (UniqueName: \"kubernetes.io/projected/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-kube-api-access-prmxm\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:57.188993 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.188988 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-maas-api-tls\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:57.189193 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.188999 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7-openshift-service-ca\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:46:57.408675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.408588 2572 generic.go:356] "Generic (PLEG): container finished" podID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" containerID="a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56" exitCode=0 Jul 10 00:46:57.408675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.408638 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" Jul 10 00:46:57.409149 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.408677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" event={"ID":"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7","Type":"ContainerDied","Data":"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56"} Jul 10 00:46:57.409149 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.408732 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56" event={"ID":"cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7","Type":"ContainerDied","Data":"cd1a4b4d4e13f5237864c28a28eba4c4ffcc09bdf764085d360ba67450642634"} Jul 10 00:46:57.409149 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.408756 2572 scope.go:117] "RemoveContainer" containerID="a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56" Jul 10 00:46:57.418041 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.418024 2572 scope.go:117] "RemoveContainer" containerID="a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56" Jul 10 00:46:57.418285 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:46:57.418267 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56\": container with ID starting with a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56 not found: ID does not exist" containerID="a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56" Jul 10 00:46:57.418329 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.418294 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56"} err="failed to get container status \"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56\": rpc error: code = NotFound desc = could not find container \"a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56\": container with ID starting with a78c550f2ab817fb4330d3cfbd289fc37b04a86ede89a4923528c7f6198e4c56 not found: ID does not exist" Jul 10 00:46:57.430991 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.430970 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:57.434766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:57.434746 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a1faa6f3-7cfbd85794-bpg56"] Jul 10 00:46:58.514233 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:46:58.514192 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" path="/var/lib/kubelet/pods/cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7/volumes" Jul 10 00:47:02.572879 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:02.572794 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:02.695295 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:02.695262 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:03.986418 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:03.986391 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:08.506972 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.506916 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:08.600426 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.600401 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:47:08.607144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.607078 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:08.609520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.609501 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:47:08.704220 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.704189 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:08.704785 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.704770 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" containerName="maas-api" Jul 10 00:47:08.704846 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.704789 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" containerName="maas-api" Jul 10 00:47:08.704939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.704891 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8cc2f8-a11b-4bc5-8887-d25a04a2c2e7" containerName="maas-api" Jul 10 00:47:08.709575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.709545 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.713822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.712717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-64a0d649-openshift-default-dockercfg-w6p9s\"" Jul 10 00:47:08.725204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.725173 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:08.734203 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.734178 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:08.789274 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789274 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789274 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789251 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789292 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789315 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jz88\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789442 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789515 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789473 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.789726 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.789569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.830820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.830790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:08.890721 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890690 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jz88\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890772 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.890891 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891237 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890921 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891237 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.890998 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891343 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.891250 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891343 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.891293 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891343 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.891311 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891483 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.891464 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.891880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.891857 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.893287 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.893264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.893626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.893609 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.899362 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.899332 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:08.899607 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:08.899593 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jz88\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88\") pod \"e2e-derive-64a0d649-openshift-default-74648d5855-4p59v\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:09.030150 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:09.030116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:09.167029 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:09.167004 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:09.168907 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:47:09.168877 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41547d25_d691_4b3e_a888_5acb85701cef.slice/crio-9305bc698ef8050c416f8c2d2324ed488d366632d43c83fcacdfe509dfb7418b WatchSource:0}: Error finding container 9305bc698ef8050c416f8c2d2324ed488d366632d43c83fcacdfe509dfb7418b: Status 404 returned error can't find the container with id 9305bc698ef8050c416f8c2d2324ed488d366632d43c83fcacdfe509dfb7418b Jul 10 00:47:09.477348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:09.477267 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" event={"ID":"41547d25-d691-4b3e-a888-5acb85701cef","Type":"ContainerStarted","Data":"9305bc698ef8050c416f8c2d2324ed488d366632d43c83fcacdfe509dfb7418b"} Jul 10 00:47:09.854465 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:09.854429 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:09.940305 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:09.940270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:10.012712 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:10.012676 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:10.169095 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:10.169014 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:10.509770 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:10.509734 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:10.611258 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:10.611219 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:11.918429 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:11.918387 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:47:11.918729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:11.918468 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:47:11.918729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:11.918501 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:47:12.013554 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:12.013521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:12.497822 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:12.497786 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" event={"ID":"41547d25-d691-4b3e-a888-5acb85701cef","Type":"ContainerStarted","Data":"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58"} Jul 10 00:47:12.516863 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:12.516819 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" podStartSLOduration=1.7705775670000001 podStartE2EDuration="4.516802295s" totalImagesPullingTime="2.746224728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:47:12.515206496 +0000 UTC m=+1804.541685214" watchObservedRunningTime="2026-07-10 00:47:12.516802295 +0000 UTC m=+1804.543280999" Jul 10 00:47:13.030302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:13.030263 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:13.035054 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:13.035029 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:13.503435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:13.503403 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:13.504360 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:13.504338 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:14.524171 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:14.524138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:14.669475 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:14.669432 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:14.690533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:14.690497 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:14.812517 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:14.812426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:14.913765 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:14.913729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:15.514370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:15.514338 2572 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" secret="" err="secret \"e2e-derive-64a0d649-openshift-default-dockercfg-w6p9s\" not found" Jul 10 00:47:16.519306 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:16.519270 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" podUID="41547d25-d691-4b3e-a888-5acb85701cef" containerName="istio-proxy" containerID="cri-o://075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58" gracePeriod=30 Jul 10 00:47:16.559640 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:16.559613 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:20.353544 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:20.353504 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:20.412280 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:20.412244 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:20.907322 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:20.907286 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:21.009177 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.009145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:21.110888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.110854 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:21.779620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.779596 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:21.910039 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.909924 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jz88\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910039 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910008 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910039 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910035 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910079 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910129 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910153 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910183 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910209 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910255 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs\") pod \"41547d25-d691-4b3e-a888-5acb85701cef\" (UID: \"41547d25-d691-4b3e-a888-5acb85701cef\") " Jul 10 00:47:21.910599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910484 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:47:21.910599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910526 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data" (OuterVolumeSpecName: "istio-data") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:47:21.910599 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910582 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:47:21.910759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910630 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:21.910759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910650 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:21.910851 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910760 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:47:21.910851 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.910776 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:47:21.912746 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.912723 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:47:21.912858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.912774 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:47:21.912858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.912838 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88" (OuterVolumeSpecName: "kube-api-access-2jz88") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "kube-api-access-2jz88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:47:21.912858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:21.912848 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token" (OuterVolumeSpecName: "istio-token") pod "41547d25-d691-4b3e-a888-5acb85701cef" (UID: "41547d25-d691-4b3e-a888-5acb85701cef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:47:22.011748 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011706 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jz88\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-kube-api-access-2jz88\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.011748 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011744 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41547d25-d691-4b3e-a888-5acb85701cef-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.011748 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011756 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41547d25-d691-4b3e-a888-5acb85701cef-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.012026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011767 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41547d25-d691-4b3e-a888-5acb85701cef-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.012026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011778 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.012026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011789 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.012026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.011799 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41547d25-d691-4b3e-a888-5acb85701cef-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:47:22.552436 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.552401 2572 generic.go:356] "Generic (PLEG): container finished" podID="41547d25-d691-4b3e-a888-5acb85701cef" containerID="075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58" exitCode=0 Jul 10 00:47:22.552583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.552441 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" Jul 10 00:47:22.552583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.552441 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" event={"ID":"41547d25-d691-4b3e-a888-5acb85701cef","Type":"ContainerDied","Data":"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58"} Jul 10 00:47:22.552583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.552493 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v" event={"ID":"41547d25-d691-4b3e-a888-5acb85701cef","Type":"ContainerDied","Data":"9305bc698ef8050c416f8c2d2324ed488d366632d43c83fcacdfe509dfb7418b"} Jul 10 00:47:22.552583 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.552515 2572 scope.go:117] "RemoveContainer" containerID="075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58" Jul 10 00:47:22.561984 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.561943 2572 scope.go:117] "RemoveContainer" containerID="075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58" Jul 10 00:47:22.562242 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:47:22.562224 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58\": container with ID starting with 075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58 not found: ID does not exist" containerID="075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58" Jul 10 00:47:22.562296 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.562250 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58"} err="failed to get container status \"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58\": rpc error: code = NotFound desc = could not find container \"075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58\": container with ID starting with 075873fb8faf6251d1d33c8d6dc9e974f1e2fd5a7f9a2fcdb7f996bc58bd3c58 not found: ID does not exist" Jul 10 00:47:22.571341 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.571320 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:22.576730 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.576711 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-64a0d649-openshift-default-74648d5855-4p59v"] Jul 10 00:47:22.776507 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:22.776421 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:24.514031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:24.513995 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41547d25-d691-4b3e-a888-5acb85701cef" path="/var/lib/kubelet/pods/41547d25-d691-4b3e-a888-5acb85701cef/volumes" Jul 10 00:47:28.466284 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:28.466245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:28.528500 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:28.528463 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:28.653128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:28.653084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:28.753107 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:28.753083 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:28.852729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:28.852696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:29.355168 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:29.355133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:47:55.497607 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:47:55.497574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:01.005431 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:01.005403 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:01.105582 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:01.105554 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:01.206988 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:01.206933 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:12.353596 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:12.353560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:12.408176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:12.408143 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:17.331707 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:17.331674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:22.890486 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:22.890448 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:22.942560 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:22.942522 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:23.044923 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:23.044877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:23.143165 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:23.143086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:33.723832 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:33.723758 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:34.677136 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:34.677099 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:40.312510 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:40.312456 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:40.374092 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:40.374055 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:40.465074 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:40.465042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:40.563319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:40.563239 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:40.662934 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:40.662899 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:41.526604 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:41.526572 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:41.624412 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:41.624382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:41.725001 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:41.724934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:43.595797 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:43.595765 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:49.177836 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:49.177802 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:49.245164 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:49.245120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:49.905072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:49.905034 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:55.473241 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:55.473203 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:55.573159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:55.573113 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:55.727751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:55.727660 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:56.063210 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.063179 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:48:56.063573 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.063559 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41547d25-d691-4b3e-a888-5acb85701cef" containerName="istio-proxy" Jul 10 00:48:56.063629 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.063576 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="41547d25-d691-4b3e-a888-5acb85701cef" containerName="istio-proxy" Jul 10 00:48:56.063664 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.063655 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="41547d25-d691-4b3e-a888-5acb85701cef" containerName="istio-proxy" Jul 10 00:48:56.066820 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.066799 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.069172 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.069151 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-23f04d4e-openshift-default-dockercfg-j847m\"" Jul 10 00:48:56.088464 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.088441 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:48:56.088670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.088646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:56.100430 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100408 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100570 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100435 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100570 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100570 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100483 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100629 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100705 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100670 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100778 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100726 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100778 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100762 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87pw8\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.100857 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.100842 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.201919 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.201882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.201919 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.201918 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.201943 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202009 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202129 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87pw8\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202216 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202339 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202363 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202423 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.202493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202448 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.203002 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.202913 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.204736 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.204709 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.204861 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.204841 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.210228 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.210211 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.210577 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.210554 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87pw8\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8\") pod \"e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.354244 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.354163 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:56.378339 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.378309 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:56.533858 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.533758 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:48:56.536386 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:48:56.536358 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d773bc2_771f_4c1d_bc10_086fb62940aa.slice/crio-87e18f4e47c8d23478936c12ef0e4e546cae7df26c27a60d36a469f3adf7318a WatchSource:0}: Error finding container 87e18f4e47c8d23478936c12ef0e4e546cae7df26c27a60d36a469f3adf7318a: Status 404 returned error can't find the container with id 87e18f4e47c8d23478936c12ef0e4e546cae7df26c27a60d36a469f3adf7318a Jul 10 00:48:56.538575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.538540 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:48:56.538659 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.538606 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:48:56.538659 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:56.538637 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:48:57.078781 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:57.078740 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerStarted","Data":"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7"} Jul 10 00:48:57.078969 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:57.078809 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerStarted","Data":"87e18f4e47c8d23478936c12ef0e4e546cae7df26c27a60d36a469f3adf7318a"} Jul 10 00:48:57.098751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:57.098699 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podStartSLOduration=1.098683221 podStartE2EDuration="1.098683221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:48:57.096221693 +0000 UTC m=+1909.122700388" watchObservedRunningTime="2026-07-10 00:48:57.098683221 +0000 UTC m=+1909.125161926" Jul 10 00:48:57.378853 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:57.378759 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:57.383596 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:57.383571 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:58.085026 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:58.084985 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:58.086047 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:58.086024 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:48:59.314405 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.314370 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:59.389738 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.389705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:59.454863 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.454824 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:59.561982 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.561934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:59.883195 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.883167 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:48:59.983545 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:48:59.983483 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:03.779284 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:03.779250 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:04.459107 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:04.459070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:04.564772 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:04.564742 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:04.953878 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:04.953843 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:10.531330 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:10.531301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:10.595023 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:10.594942 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:11.120883 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:11.120849 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:11.463316 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:11.463244 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:11.523350 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:11.523300 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:12.178674 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.178637 2572 generic.go:356] "Generic (PLEG): container finished" podID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerID="c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7" exitCode=0 Jul 10 00:49:12.179128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.178709 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerDied","Data":"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7"} Jul 10 00:49:12.179174 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.179146 2572 scope.go:117] "RemoveContainer" containerID="c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7" Jul 10 00:49:12.180237 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.180211 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:12.180314 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.180297 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:12.180358 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:12.180345 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:13.085337 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:13.085285 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:13.186100 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:13.186060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerStarted","Data":"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3"} Jul 10 00:49:13.379549 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:13.379452 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:13.380763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:13.380734 2572 patch_prober.go:28] interesting pod/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 10 00:49:13.380908 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:13.380793 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 10 00:49:14.379565 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:14.379524 2572 patch_prober.go:28] interesting pod/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 10 00:49:14.379977 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:14.379578 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 10 00:49:15.378851 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:15.378811 2572 patch_prober.go:28] interesting pod/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 10 00:49:15.379099 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:15.378873 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 10 00:49:16.378922 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:16.378885 2572 patch_prober.go:28] interesting pod/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 10 00:49:16.379352 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:16.378977 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 10 00:49:17.383084 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.383056 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:17.383552 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.383498 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:17.384100 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.384082 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:17.578620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.578574 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:17.633965 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.633859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:17.822654 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.822615 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:49:17.838614 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:17.838583 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:18.075426 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:18.075398 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:18.213374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:18.213345 2572 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" secret="" err="secret \"e2e-mt-23f04d4e-openshift-default-dockercfg-j847m\" not found" Jul 10 00:49:18.219305 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:18.219272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:18.311469 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:18.311436 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:18.433392 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:18.433313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:19.218881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:19.218823 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" containerID="cri-o://43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3" gracePeriod=30 Jul 10 00:49:24.186239 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.186195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:24.289312 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.289282 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:24.385176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.385144 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:24.478375 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.478346 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:24.551593 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551563 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.551737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551603 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.551737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551667 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.551737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551687 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87pw8\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.551737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551715 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.551737 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551731 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.552065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551784 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.552065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551841 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.552065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.551888 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs\") pod \"5d773bc2-771f-4c1d-bc10-086fb62940aa\" (UID: \"5d773bc2-771f-4c1d-bc10-086fb62940aa\") " Jul 10 00:49:24.552217 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552066 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:49:24.552271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552199 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:49:24.552271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552253 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.552271 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552257 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:49:24.552555 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552483 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:49:24.552691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.552582 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:49:24.553857 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.553832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:24.554340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.554318 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:49:24.554435 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.554398 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:49:24.554484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.554449 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8" (OuterVolumeSpecName: "kube-api-access-87pw8") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "kube-api-access-87pw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:49:24.554644 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.554618 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d773bc2-771f-4c1d-bc10-086fb62940aa" (UID: "5d773bc2-771f-4c1d-bc10-086fb62940aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:49:24.652813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652786 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.652813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652811 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87pw8\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-kube-api-access-87pw8\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652822 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652831 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d773bc2-771f-4c1d-bc10-086fb62940aa-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652839 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652847 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652855 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:24.653027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:24.652862 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d773bc2-771f-4c1d-bc10-086fb62940aa-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:49:25.253390 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.253359 2572 generic.go:356] "Generic (PLEG): container finished" podID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerID="43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3" exitCode=0 Jul 10 00:49:25.253813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.253422 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerDied","Data":"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3"} Jul 10 00:49:25.253813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.253462 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" event={"ID":"5d773bc2-771f-4c1d-bc10-086fb62940aa","Type":"ContainerDied","Data":"87e18f4e47c8d23478936c12ef0e4e546cae7df26c27a60d36a469f3adf7318a"} Jul 10 00:49:25.253813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.253426 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth" Jul 10 00:49:25.253813 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.253482 2572 scope.go:117] "RemoveContainer" containerID="43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3" Jul 10 00:49:25.263235 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.263219 2572 scope.go:117] "RemoveContainer" containerID="c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7" Jul 10 00:49:25.271391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.271370 2572 scope.go:117] "RemoveContainer" containerID="43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3" Jul 10 00:49:25.271648 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:49:25.271631 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3\": container with ID starting with 43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3 not found: ID does not exist" containerID="43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3" Jul 10 00:49:25.271688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.271657 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3"} err="failed to get container status \"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3\": rpc error: code = NotFound desc = could not find container \"43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3\": container with ID starting with 43ce9784ef482573ae149bbb506f24abfa16bf9eae958d97db0666fc8f0705d3 not found: ID does not exist" Jul 10 00:49:25.271688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.271673 2572 scope.go:117] "RemoveContainer" containerID="c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7" Jul 10 00:49:25.271892 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:49:25.271875 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7\": container with ID starting with c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7 not found: ID does not exist" containerID="c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7" Jul 10 00:49:25.271930 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.271900 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7"} err="failed to get container status \"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7\": rpc error: code = NotFound desc = could not find container \"c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7\": container with ID starting with c4766cd733d0ea892c713f0946f364e67cae4787c17d7c010eb13aea1c29e0d7 not found: ID does not exist" Jul 10 00:49:25.275324 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.275298 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:49:25.278715 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:25.278696 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-23f04d4e-openshift-default-7768b5784c-95mth"] Jul 10 00:49:26.513862 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:26.513832 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" path="/var/lib/kubelet/pods/5d773bc2-771f-4c1d-bc10-086fb62940aa/volumes" Jul 10 00:49:28.586219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:28.586182 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:28.688304 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:28.688272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:30.179807 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:30.179770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:30.280252 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:30.280220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:37.141666 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.141631 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:37.435860 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.435770 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:49:37.436375 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436356 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.436454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436378 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.436454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436394 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.436454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436403 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.436614 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436549 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.436614 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.436563 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d773bc2-771f-4c1d-bc10-086fb62940aa" containerName="istio-proxy" Jul 10 00:49:37.441234 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.441213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.443484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.443465 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-de57c55c-openshift-default-dockercfg-zvhtp\"" Jul 10 00:49:37.449049 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.448829 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:49:37.472043 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.472017 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:37.562853 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.562822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.562860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.562882 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.562975 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.563031 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563179 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.563062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563179 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.563091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563179 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.563122 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kf8r\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.563179 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.563168 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664033 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.663996 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664216 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664216 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664098 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664216 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kf8r\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664216 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664401 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664471 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.664780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.664662 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.665159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.665121 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.666857 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.666830 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.667019 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.667003 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.681405 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.681384 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.681708 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.681682 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kf8r\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r\") pod \"e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.717769 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.717689 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:37.752622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.752593 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:37.896035 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.896014 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:49:37.897428 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:49:37.897401 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5735e8c6_7994_4c45_b79f_1489fe89ae0d.slice/crio-7052cf951e36533b6a31b37e8bf5a12923ba7c9d286b7f54511b6c141cd2b60d WatchSource:0}: Error finding container 7052cf951e36533b6a31b37e8bf5a12923ba7c9d286b7f54511b6c141cd2b60d: Status 404 returned error can't find the container with id 7052cf951e36533b6a31b37e8bf5a12923ba7c9d286b7f54511b6c141cd2b60d Jul 10 00:49:37.899526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.899497 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:37.899601 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.899558 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:37.899601 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:37.899586 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:38.327157 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:38.327119 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerStarted","Data":"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164"} Jul 10 00:49:38.327549 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:38.327174 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerStarted","Data":"7052cf951e36533b6a31b37e8bf5a12923ba7c9d286b7f54511b6c141cd2b60d"} Jul 10 00:49:38.347927 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:38.347884 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podStartSLOduration=1.34786912 podStartE2EDuration="1.34786912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:49:38.345377382 +0000 UTC m=+1950.371856085" watchObservedRunningTime="2026-07-10 00:49:38.34786912 +0000 UTC m=+1950.374347823" Jul 10 00:49:38.752797 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:38.752762 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:38.757652 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:38.757625 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:39.333349 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:39.333316 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:39.334410 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:39.334388 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:40.281768 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:40.281731 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:40.379373 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:40.379341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:40.536472 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:40.536387 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:40.922467 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:40.922367 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:41.022553 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:41.022521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:41.122817 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:41.122776 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:44.804141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:44.804106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:54.646627 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:54.645185 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:34216->10.132.0.51:15021: read: connection reset by peer" start-of-body= Jul 10 00:49:54.646627 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:54.645270 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:34216->10.132.0.51:15021: read: connection reset by peer" Jul 10 00:49:54.921029 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:54.920920 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:54.970809 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:54.970777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:49:55.425627 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.425593 2572 generic.go:356] "Generic (PLEG): container finished" podID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerID="b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164" exitCode=0 Jul 10 00:49:55.425835 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.425671 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerDied","Data":"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164"} Jul 10 00:49:55.426145 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.426126 2572 scope.go:117] "RemoveContainer" containerID="b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164" Jul 10 00:49:55.427123 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.427099 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:55.427186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.427164 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:55.427222 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:55.427193 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:49:56.442031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:56.441993 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerStarted","Data":"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602"} Jul 10 00:49:56.753810 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:56.753786 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:49:56.754039 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:56.754017 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 00:49:56.754091 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:56.754066 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 00:49:57.754127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:57.754090 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 00:49:57.754626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:57.754152 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 00:49:58.753588 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:58.753547 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 00:49:58.753811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:58.753626 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 00:49:59.753866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:59.753774 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 10 00:49:59.753866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:49:59.753831 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 10 00:50:00.757066 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:00.757034 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:50:00.757525 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:00.757303 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:50:00.757880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:00.757863 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:50:06.398916 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:06.398832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:50:32.697605 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:32.697571 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:50:33.048625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:33.048594 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:50:38.811605 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:38.811561 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:50:43.919043 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:50:43.919006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:13.336575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:13.336534 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:20.487853 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:20.487813 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:20.788594 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:20.788566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:26.381728 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.381690 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:26.441509 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.441469 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:26.649864 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.649785 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:51:26.650070 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.650044 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" containerID="cri-o://cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602" gracePeriod=30 Jul 10 00:51:26.681454 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.681414 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:26.910132 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:26.910011 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:27.999537 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:27.999505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:28.100780 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:28.100747 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:28.198377 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:28.198343 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:28.947996 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:28.947934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:29.046566 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:29.046533 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:29.151683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:29.151641 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:30.658088 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:30.657974 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:30.759764 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:30.759733 2572 patch_prober.go:28] interesting pod/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 00:51:30.759924 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:30.759791 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 00:51:31.890675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.890649 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:51:31.891685 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891668 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.891750 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891734 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kf8r\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.891788 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891768 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.891880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891868 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.891932 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891890 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.891932 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891910 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.892062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891936 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.892062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891864 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:51:31.892062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.891978 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.892062 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892042 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data\") pod \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\" (UID: \"5735e8c6-7994-4c45-b79f-1489fe89ae0d\") " Jul 10 00:51:31.892245 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892134 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:51:31.892299 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892257 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:51:31.892424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892406 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.892508 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892436 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.892508 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892451 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.892508 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.892400 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:51:31.893325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.893299 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:51:31.894405 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.894367 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:51:31.894504 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.894404 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r" (OuterVolumeSpecName: "kube-api-access-5kf8r") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "kube-api-access-5kf8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:51:31.894579 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.894534 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:51:31.894579 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.894534 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5735e8c6-7994-4c45-b79f-1489fe89ae0d" (UID: "5735e8c6-7994-4c45-b79f-1489fe89ae0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:51:31.966532 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.966438 2572 generic.go:356] "Generic (PLEG): container finished" podID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerID="cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602" exitCode=0 Jul 10 00:51:31.966532 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.966485 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerDied","Data":"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602"} Jul 10 00:51:31.966532 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.966524 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" event={"ID":"5735e8c6-7994-4c45-b79f-1489fe89ae0d","Type":"ContainerDied","Data":"7052cf951e36533b6a31b37e8bf5a12923ba7c9d286b7f54511b6c141cd2b60d"} Jul 10 00:51:31.966763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.966545 2572 scope.go:117] "RemoveContainer" containerID="cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602" Jul 10 00:51:31.966763 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.966487 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9" Jul 10 00:51:31.976512 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.976493 2572 scope.go:117] "RemoveContainer" containerID="b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164" Jul 10 00:51:31.984976 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.984938 2572 scope.go:117] "RemoveContainer" containerID="cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602" Jul 10 00:51:31.985327 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:51:31.985301 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602\": container with ID starting with cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602 not found: ID does not exist" containerID="cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602" Jul 10 00:51:31.985472 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.985449 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602"} err="failed to get container status \"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602\": rpc error: code = NotFound desc = could not find container \"cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602\": container with ID starting with cdd92678d08c9147998dfbdb673f4afe768c6e97c235d8f1eb1a68806f27a602 not found: ID does not exist" Jul 10 00:51:31.985581 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.985562 2572 scope.go:117] "RemoveContainer" containerID="b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164" Jul 10 00:51:31.985885 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:51:31.985860 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164\": container with ID starting with b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164 not found: ID does not exist" containerID="b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164" Jul 10 00:51:31.986308 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.985895 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164"} err="failed to get container status \"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164\": rpc error: code = NotFound desc = could not find container \"b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164\": container with ID starting with b0e1838ca4e7db26a902d7442b137db01fc627cc95e320ed88aef17880f3c164 not found: ID does not exist" Jul 10 00:51:31.988144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.988125 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:51:31.992211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.992191 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-de57c55c-openshift-default-7b745b44cf-w6bx9"] Jul 10 00:51:31.993064 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993045 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.993114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993068 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.993114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993079 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.993114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993087 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.993114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993094 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735e8c6-7994-4c45-b79f-1489fe89ae0d-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:31.993114 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:31.993102 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kf8r\" (UniqueName: \"kubernetes.io/projected/5735e8c6-7994-4c45-b79f-1489fe89ae0d-kube-api-access-5kf8r\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:51:32.513819 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:32.513781 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" path="/var/lib/kubelet/pods/5735e8c6-7994-4c45-b79f-1489fe89ae0d/volumes" Jul 10 00:51:36.601204 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:36.601165 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:36.701560 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:36.701526 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:37.312545 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:37.312511 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:43.049667 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:43.049633 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:43.118132 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:43.118107 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:43.208438 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:43.208405 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:43.308631 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:43.308524 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:53.800067 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:53.800031 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:51:54.171782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:51:54.171674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:08.643305 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:08.643275 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:52:08.650855 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:08.650833 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:52:15.158614 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:15.158579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:20.715845 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:20.715814 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:20.770548 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:20.770514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:21.070216 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.070182 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:21.387395 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387316 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:52:21.387720 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387707 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.387720 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387720 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.387828 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387734 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.387828 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387741 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.387828 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387815 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.387828 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.387823 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5735e8c6-7994-4c45-b79f-1489fe89ae0d" containerName="istio-proxy" Jul 10 00:52:21.390966 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.390932 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.393364 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.393345 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3ea0d8-openshift-default-dockercfg-jqlmp\"" Jul 10 00:52:21.400901 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.400879 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:52:21.408338 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.408316 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:21.527370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527341 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd4j4\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527498 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527516 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527676 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527535 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527676 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527572 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527676 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527615 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.527676 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.527651 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.628824 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628832 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628873 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd4j4\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628892 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628908 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628931 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629013 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.628997 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629061 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629465 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629426 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629465 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629433 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629577 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629531 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629585 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.629805 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.629781 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.631298 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.631281 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.631587 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.631566 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.645778 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.645718 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd4j4\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.645778 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.645730 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token\") pod \"e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.676978 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.676936 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:21.703209 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.703186 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:21.845021 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.844993 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:52:21.846092 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:52:21.846067 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba138ef_7236_4864_82af_1d73406fb35f.slice/crio-3585f9141bf6446a789ff9a06fde8576442ee7c5d03c5fa27dfc642be9acb515 WatchSource:0}: Error finding container 3585f9141bf6446a789ff9a06fde8576442ee7c5d03c5fa27dfc642be9acb515: Status 404 returned error can't find the container with id 3585f9141bf6446a789ff9a06fde8576442ee7c5d03c5fa27dfc642be9acb515 Jul 10 00:52:21.848095 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.848062 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:21.848200 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.848140 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:21.848263 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:21.848206 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:22.241734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:22.241649 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerStarted","Data":"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98"} Jul 10 00:52:22.241734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:22.241700 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerStarted","Data":"3585f9141bf6446a789ff9a06fde8576442ee7c5d03c5fa27dfc642be9acb515"} Jul 10 00:52:22.260931 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:22.260879 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podStartSLOduration=1.2608637329999999 podStartE2EDuration="1.260863733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:52:22.258184734 +0000 UTC m=+2114.284663438" watchObservedRunningTime="2026-07-10 00:52:22.260863733 +0000 UTC m=+2114.287342436" Jul 10 00:52:22.703580 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:22.703546 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:22.708495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:22.708471 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:23.247063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:23.247029 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:23.248002 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:23.247977 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:24.275123 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:24.275088 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:24.346997 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:24.346909 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:24.415054 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:24.415027 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:24.834863 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:24.834828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:24.934961 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:24.934926 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:25.037688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:25.037657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:28.831619 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:28.831562 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:29.200128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.200044 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:29.480325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.480245 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:52:29.484882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.484863 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.487221 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.487205 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-50269d-openshift-default-dockercfg-nb9v8\"" Jul 10 00:52:29.491617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.491588 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.491798 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.491778 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.491920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.491904 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492138 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492119 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492277 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492254 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492277 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492427 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.492881 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.492463 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgk22\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.495594 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.495573 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:52:29.511414 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.511385 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:29.593357 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.593546 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.593546 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593424 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgk22\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.593546 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593491 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.593805 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593725 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.593992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594181 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594163 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594232 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594261 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594365 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594521 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594476 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594554 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594597 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.594794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.594781 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.596930 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.596904 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.597080 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.597062 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.600696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.600673 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.600961 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.600930 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgk22\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22\") pod \"e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.796686 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.796653 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:29.844522 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.844439 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:29.942230 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.942207 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:52:29.944259 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:52:29.944228 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa177a19_962d_4b7b_bcb2_f2bc3eb1cfc5.slice/crio-4487c410f67dd62e5d5928737b433404d51fbc6edf6ef21e4c8c445873c5a658 WatchSource:0}: Error finding container 4487c410f67dd62e5d5928737b433404d51fbc6edf6ef21e4c8c445873c5a658: Status 404 returned error can't find the container with id 4487c410f67dd62e5d5928737b433404d51fbc6edf6ef21e4c8c445873c5a658 Jul 10 00:52:29.946451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.946424 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:29.946535 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.946484 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:29.946535 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:29.946517 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:30.286443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:30.286399 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerStarted","Data":"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165"} Jul 10 00:52:30.286589 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:30.286470 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerStarted","Data":"4487c410f67dd62e5d5928737b433404d51fbc6edf6ef21e4c8c445873c5a658"} Jul 10 00:52:30.306417 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:30.306366 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podStartSLOduration=1.306351399 podStartE2EDuration="1.306351399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:52:30.304427239 +0000 UTC m=+2122.330905980" watchObservedRunningTime="2026-07-10 00:52:30.306351399 +0000 UTC m=+2122.332830102" Jul 10 00:52:30.797141 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:30.797111 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:30.802156 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:30.802129 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:31.292002 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:31.291967 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:31.293009 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:31.292988 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:32.244638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.244600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:32.339426 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.339392 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:32.406083 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.406048 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:32.504346 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.504302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:32.818049 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.816940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:32.911286 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:32.911255 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:36.765118 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:36.765063 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:39.289183 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:39.289125 2572 patch_prober.go:28] interesting pod/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 00:52:39.289620 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:39.289248 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 00:52:40.350031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.349995 2572 generic.go:356] "Generic (PLEG): container finished" podID="2ba138ef-7236-4864-82af-1d73406fb35f" containerID="0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98" exitCode=0 Jul 10 00:52:40.350444 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.350073 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerDied","Data":"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98"} Jul 10 00:52:40.350542 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.350524 2572 scope.go:117] "RemoveContainer" containerID="0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98" Jul 10 00:52:40.351608 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.351583 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:40.351677 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.351649 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:40.351715 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:40.351680 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:41.362226 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:41.362182 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerStarted","Data":"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba"} Jul 10 00:52:41.704105 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:41.704022 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:41.704346 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:41.704238 2572 patch_prober.go:28] interesting pod/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 00:52:41.704407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:41.704375 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 00:52:42.703582 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:42.703546 2572 patch_prober.go:28] interesting pod/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 00:52:42.704072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:42.703988 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 00:52:43.703759 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:43.703725 2572 patch_prober.go:28] interesting pod/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 00:52:43.704112 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:43.703788 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 00:52:44.703696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:44.703658 2572 patch_prober.go:28] interesting pod/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 00:52:44.703877 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:44.703722 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 00:52:45.706925 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:45.706891 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:45.707407 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:45.707288 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:45.708012 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:45.707991 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:52:46.292688 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:46.292633 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:46.396683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:46.396648 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:46.774887 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:46.774170 2572 patch_prober.go:28] interesting pod/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:39232->10.132.0.53:15021: read: connection reset by peer" start-of-body= Jul 10 00:52:46.774887 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:46.774242 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:39232->10.132.0.53:15021: read: connection reset by peer" Jul 10 00:52:47.402484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.402452 2572 generic.go:356] "Generic (PLEG): container finished" podID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerID="19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165" exitCode=0 Jul 10 00:52:47.402675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.402526 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerDied","Data":"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165"} Jul 10 00:52:47.403020 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.402989 2572 scope.go:117] "RemoveContainer" containerID="19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165" Jul 10 00:52:47.403940 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.403915 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:47.404036 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.404015 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:47.404103 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:47.404047 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:52:48.410051 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:48.409985 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerStarted","Data":"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9"} Jul 10 00:52:48.797716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:48.797676 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:48.798051 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:48.798028 2572 patch_prober.go:28] interesting pod/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 00:52:48.798126 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:48.798095 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 00:52:49.530648 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:49.530616 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:49.797463 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:49.797369 2572 patch_prober.go:28] interesting pod/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 00:52:49.797463 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:49.797427 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 00:52:50.797439 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:50.797408 2572 patch_prober.go:28] interesting pod/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 00:52:50.797804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:50.797463 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 00:52:51.797144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:51.797106 2572 patch_prober.go:28] interesting pod/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 10 00:52:51.797324 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:51.797174 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 10 00:52:52.801408 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:52.801375 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:52.801878 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:52.801773 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:52.802517 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:52.802497 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:52:53.502675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:53.502641 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:52:53.603598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:52:53.603567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:29.700848 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:29.700810 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:50.778456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:50.778425 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:51.704540 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:51.704504 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:51.856293 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:51.856258 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:52.289135 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:52.289102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:52.390738 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:52.390701 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:52.490807 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:52.490772 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:52.596291 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:52.596207 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:53:57.362524 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:53:57.362491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:29.350725 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:29.350684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:29.451503 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:29.451470 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:34.181480 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.181449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:34.453389 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.453307 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:54:34.453560 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.453538 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" containerID="cri-o://3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba" gracePeriod=30 Jul 10 00:54:34.476456 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.476426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:34.722328 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.722237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:34.906077 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:34.906044 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:35.325680 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:35.325644 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:35.426675 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:35.426640 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:35.575348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:35.575313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:39.712052 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.711989 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:54:39.730559 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730533 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730574 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730631 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730716 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730711 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730741 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730766 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730812 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.730880 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730869 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd4j4\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.731099 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730904 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token\") pod \"2ba138ef-7236-4864-82af-1d73406fb35f\" (UID: \"2ba138ef-7236-4864-82af-1d73406fb35f\") " Jul 10 00:54:39.731099 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.730979 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:54:39.731200 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.731129 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:39.732131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.731275 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.732131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.731297 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba138ef-7236-4864-82af-1d73406fb35f-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.732131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.731892 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:39.732131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.732090 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:39.732467 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.732133 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:39.733626 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.733586 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:39.733817 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.733788 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:54:39.733916 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.733903 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:54:39.734191 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.734160 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4" (OuterVolumeSpecName: "kube-api-access-qd4j4") pod "2ba138ef-7236-4864-82af-1d73406fb35f" (UID: "2ba138ef-7236-4864-82af-1d73406fb35f"). InnerVolumeSpecName "kube-api-access-qd4j4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:54:39.832165 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832138 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832165 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832166 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba138ef-7236-4864-82af-1d73406fb35f-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832180 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832189 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832198 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd4j4\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-kube-api-access-qd4j4\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832206 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba138ef-7236-4864-82af-1d73406fb35f-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:39.832361 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:39.832213 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba138ef-7236-4864-82af-1d73406fb35f-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:40.052224 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.052186 2572 generic.go:356] "Generic (PLEG): container finished" podID="2ba138ef-7236-4864-82af-1d73406fb35f" containerID="3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba" exitCode=0 Jul 10 00:54:40.052391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.052237 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" Jul 10 00:54:40.052391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.052276 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerDied","Data":"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba"} Jul 10 00:54:40.052391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.052330 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg" event={"ID":"2ba138ef-7236-4864-82af-1d73406fb35f","Type":"ContainerDied","Data":"3585f9141bf6446a789ff9a06fde8576442ee7c5d03c5fa27dfc642be9acb515"} Jul 10 00:54:40.052391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.052355 2572 scope.go:117] "RemoveContainer" containerID="3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba" Jul 10 00:54:40.062785 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.062767 2572 scope.go:117] "RemoveContainer" containerID="0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98" Jul 10 00:54:40.071410 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.071392 2572 scope.go:117] "RemoveContainer" containerID="3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba" Jul 10 00:54:40.071677 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:54:40.071649 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba\": container with ID starting with 3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba not found: ID does not exist" containerID="3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba" Jul 10 00:54:40.071782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.071687 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba"} err="failed to get container status \"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba\": rpc error: code = NotFound desc = could not find container \"3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba\": container with ID starting with 3a78041101754328faf3dff2a0fcbf25206f8cdbb8e6ee08ef09f834f26f58ba not found: ID does not exist" Jul 10 00:54:40.071782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.071705 2572 scope.go:117] "RemoveContainer" containerID="0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98" Jul 10 00:54:40.072219 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:54:40.072194 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98\": container with ID starting with 0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98 not found: ID does not exist" containerID="0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98" Jul 10 00:54:40.072330 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.072227 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98"} err="failed to get container status \"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98\": rpc error: code = NotFound desc = could not find container \"0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98\": container with ID starting with 0cedd7f99d3877518c19b182c9cefcc91ad1dbc6030ef55a7794716f81665d98 not found: ID does not exist" Jul 10 00:54:40.075234 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.075210 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:54:40.080994 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.080973 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3ea0d8-openshift-default-d8bdd9f5f-xh4bg"] Jul 10 00:54:40.514145 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:40.514115 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" path="/var/lib/kubelet/pods/2ba138ef-7236-4864-82af-1d73406fb35f/volumes" Jul 10 00:54:41.126427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.126395 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:41.186032 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.185986 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:41.391673 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.391571 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:54:41.391872 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.391843 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" containerID="cri-o://5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9" gracePeriod=30 Jul 10 00:54:41.411897 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.411865 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:41.646855 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:41.646773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:42.016014 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.015981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:42.306718 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.306641 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:54:42.307131 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.307116 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:42.307175 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.307132 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:42.307211 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.307205 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:42.311735 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.311716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.314189 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.314166 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3f46d3-openshift-default-dockercfg-rmlq5\"" Jul 10 00:54:42.320918 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.320894 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:54:42.328409 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.328381 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:42.355843 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.355813 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.355996 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.355879 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.355996 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.355912 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356082 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356082 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356072 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356182 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356182 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356139 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356182 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356163 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.356182 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.356182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkvc5\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457032 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457077 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457127 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457184 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkvc5\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457270 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457236 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457326 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457452 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457617 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457656 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.457815 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.457722 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.458112 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.458091 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.459693 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.459677 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.459836 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.459820 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.465295 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.465275 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkvc5\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.465382 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.465305 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token\") pod \"e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.565596 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.565506 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:42.624575 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.624542 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:42.765882 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.765842 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:54:42.769127 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:54:42.769096 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode899e0c5_0029_4485_9c12_08d4f1a82059.slice/crio-4c1a5de2b5c3cab9fd71520bed251ca33692e4725fce835987877215fd16a304 WatchSource:0}: Error finding container 4c1a5de2b5c3cab9fd71520bed251ca33692e4725fce835987877215fd16a304: Status 404 returned error can't find the container with id 4c1a5de2b5c3cab9fd71520bed251ca33692e4725fce835987877215fd16a304 Jul 10 00:54:42.771679 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.771640 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:42.771795 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.771714 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:42.771795 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:42.771750 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:43.073782 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:43.073742 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerStarted","Data":"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d"} Jul 10 00:54:43.073985 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:43.073807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerStarted","Data":"4c1a5de2b5c3cab9fd71520bed251ca33692e4725fce835987877215fd16a304"} Jul 10 00:54:43.099032 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:43.098879 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podStartSLOduration=1.098863586 podStartE2EDuration="1.098863586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:54:43.097811873 +0000 UTC m=+2255.124290582" watchObservedRunningTime="2026-07-10 00:54:43.098863586 +0000 UTC m=+2255.125342283" Jul 10 00:54:43.625029 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:43.624988 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:43.629866 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:43.629840 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:44.079190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:44.079164 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:44.080221 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:44.080200 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:54:45.059523 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:45.059482 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:45.160169 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:45.160134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:45.263443 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:45.263408 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:45.802453 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:45.802421 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:45.900674 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:45.900634 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:46.358700 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.358665 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:46.651933 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.651904 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:54:46.694426 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694397 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694426 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694432 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694463 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694494 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694521 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694582 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694636 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgk22\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694717 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:46.694974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694726 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694756 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo\") pod \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\" (UID: \"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5\") " Jul 10 00:54:46.694974 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.694920 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:46.695205 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.695091 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.695205 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.695110 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.695374 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.695352 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:46.695496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.695428 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:46.695496 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.695428 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:54:46.696804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.696781 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:54:46.697462 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.697441 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22" (OuterVolumeSpecName: "kube-api-access-jgk22") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "kube-api-access-jgk22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:54:46.697588 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.697452 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:54:46.697588 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.697471 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" (UID: "fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:54:46.795762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795726 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.795762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795754 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.795762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795763 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.795762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795771 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.796063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795779 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.796063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795786 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:46.796063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:46.795794 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgk22\" (UniqueName: \"kubernetes.io/projected/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5-kube-api-access-jgk22\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:54:47.102622 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.102584 2572 generic.go:356] "Generic (PLEG): container finished" podID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerID="5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9" exitCode=0 Jul 10 00:54:47.102801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.102646 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" Jul 10 00:54:47.102801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.102667 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerDied","Data":"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9"} Jul 10 00:54:47.102801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.102763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj" event={"ID":"fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5","Type":"ContainerDied","Data":"4487c410f67dd62e5d5928737b433404d51fbc6edf6ef21e4c8c445873c5a658"} Jul 10 00:54:47.102801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.102793 2572 scope.go:117] "RemoveContainer" containerID="5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9" Jul 10 00:54:47.113176 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.113143 2572 scope.go:117] "RemoveContainer" containerID="19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165" Jul 10 00:54:47.122159 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.122002 2572 scope.go:117] "RemoveContainer" containerID="5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9" Jul 10 00:54:47.122438 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:54:47.122417 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9\": container with ID starting with 5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9 not found: ID does not exist" containerID="5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9" Jul 10 00:54:47.122518 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.122449 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9"} err="failed to get container status \"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9\": rpc error: code = NotFound desc = could not find container \"5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9\": container with ID starting with 5b7a5b7b07c3f111ed68ad7ffb22c2744bd9958a1d17a14b2cfe1d79432b4dd9 not found: ID does not exist" Jul 10 00:54:47.122518 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.122471 2572 scope.go:117] "RemoveContainer" containerID="19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165" Jul 10 00:54:47.122704 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:54:47.122687 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165\": container with ID starting with 19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165 not found: ID does not exist" containerID="19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165" Jul 10 00:54:47.122750 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.122712 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165"} err="failed to get container status \"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165\": rpc error: code = NotFound desc = could not find container \"19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165\": container with ID starting with 19c5d0b30ed9dfd9aec5289a73dbedb3308e32ee5b46ee0084e835036b7e3165 not found: ID does not exist" Jul 10 00:54:47.124314 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.124294 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:54:47.127876 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:47.127857 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-50269d-openshift-default-6d94c89cbc-9l4gj"] Jul 10 00:54:48.514745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:48.514702 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" path="/var/lib/kubelet/pods/fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5/volumes" Jul 10 00:54:49.579227 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:49.579197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:50.055323 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.055290 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:50.349753 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.349657 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:54:50.350153 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350138 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:54:50.350208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350155 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:54:50.350208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350175 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:50.350208 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350180 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:50.350297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350247 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ba138ef-7236-4864-82af-1d73406fb35f" containerName="istio-proxy" Jul 10 00:54:50.350297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350258 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:54:50.350297 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.350267 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:54:50.354690 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.354668 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.356944 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.356924 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-08071d-openshift-default-dockercfg-t6gqm\"" Jul 10 00:54:50.366147 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.366120 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:54:50.374144 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.374115 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:50.431621 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431668 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431811 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431770 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm68r\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431817 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.431939 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431909 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.432124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.431982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.432124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.432046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.533358 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.533358 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.533598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533400 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.533598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm68r\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.533598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533479 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533844 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533902 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533935 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534017 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533982 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.534198 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.534208 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.533932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534290 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.534288 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.534543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.534519 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.536437 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.536419 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.536659 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.536641 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.541038 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.541014 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm68r\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.541175 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.541148 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token\") pod \"e2e-api-b-08071d-openshift-default-c98db8c45-sf896\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.613692 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.613610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:50.666796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.666764 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:50.813261 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.813235 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:54:50.815652 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:54:50.815614 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6f373c_fc14_4c7b_80d6_fa6dd43d6f43.slice/crio-948a01fc1be4633a8b6511980d09f1c6300c3a3d5c4fe0c0cf6170498153c59b WatchSource:0}: Error finding container 948a01fc1be4633a8b6511980d09f1c6300c3a3d5c4fe0c0cf6170498153c59b: Status 404 returned error can't find the container with id 948a01fc1be4633a8b6511980d09f1c6300c3a3d5c4fe0c0cf6170498153c59b Jul 10 00:54:50.818112 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.818074 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:50.818184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.818172 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:50.818238 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:50.818216 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:54:51.129000 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:51.128895 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerStarted","Data":"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0"} Jul 10 00:54:51.129000 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:51.128974 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerStarted","Data":"948a01fc1be4633a8b6511980d09f1c6300c3a3d5c4fe0c0cf6170498153c59b"} Jul 10 00:54:51.154319 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:51.154267 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podStartSLOduration=1.154251852 podStartE2EDuration="1.154251852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:54:51.151340856 +0000 UTC m=+2263.177819560" watchObservedRunningTime="2026-07-10 00:54:51.154251852 +0000 UTC m=+2263.180730556" Jul 10 00:54:51.667284 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:51.667241 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:51.672344 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:51.672316 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:52.136247 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:52.136213 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:52.137150 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:52.137125 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:54:53.082643 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.082614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:53.161060 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.161030 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:53.233533 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.233499 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:53.352293 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.352202 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:53.684154 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.684072 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:53.784161 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:53.784121 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:54:57.625260 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:54:57.625214 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:00.121192 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:00.121134 2572 patch_prober.go:28] interesting pod/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 00:55:00.121598 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:00.121236 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 00:55:02.193914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.193878 2572 generic.go:356] "Generic (PLEG): container finished" podID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerID="4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d" exitCode=0 Jul 10 00:55:02.194343 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.193965 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerDied","Data":"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d"} Jul 10 00:55:02.194388 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.194358 2572 scope.go:117] "RemoveContainer" containerID="4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d" Jul 10 00:55:02.195444 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.195420 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:02.195524 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.195511 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:02.195561 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:02.195543 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:03.201340 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:03.201300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerStarted","Data":"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66"} Jul 10 00:55:03.625978 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:03.625923 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:03.626191 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:03.626082 2572 patch_prober.go:28] interesting pod/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 00:55:03.626191 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:03.626149 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 00:55:04.629373 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:04.629337 2572 patch_prober.go:28] interesting pod/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 00:55:04.630005 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:04.629940 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 00:55:05.625345 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:05.625309 2572 patch_prober.go:28] interesting pod/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 00:55:05.625684 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:05.625631 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 00:55:05.869729 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:05.869694 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:06.263888 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.263797 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:06.367190 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.367158 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:06.464985 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.464900 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:06.592817 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.592713 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:06.625484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.625007 2572 patch_prober.go:28] interesting pod/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 10 00:55:06.625484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:06.625084 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 10 00:55:07.628674 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:07.628639 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:07.629199 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:07.628898 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:07.629747 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:07.629721 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:08.179128 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:08.179092 2572 patch_prober.go:28] interesting pod/e2e-api-b-08071d-openshift-default-c98db8c45-sf896 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 00:55:08.179320 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:08.179191 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 00:55:09.240030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.240002 2572 generic.go:356] "Generic (PLEG): container finished" podID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerID="c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0" exitCode=0 Jul 10 00:55:09.240391 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.240080 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerDied","Data":"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0"} Jul 10 00:55:09.240613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.240596 2572 scope.go:117] "RemoveContainer" containerID="c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0" Jul 10 00:55:09.241572 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.241547 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:09.241685 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.241623 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:09.241754 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:09.241716 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:10.249127 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:10.249091 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerStarted","Data":"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004"} Jul 10 00:55:10.667219 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:10.667182 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:10.667399 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:10.667367 2572 patch_prober.go:28] interesting pod/e2e-api-b-08071d-openshift-default-c98db8c45-sf896 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 00:55:10.667445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:10.667417 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 00:55:11.667381 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:11.667341 2572 patch_prober.go:28] interesting pod/e2e-api-b-08071d-openshift-default-c98db8c45-sf896 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 00:55:11.667848 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:11.667400 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 00:55:12.264333 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.264300 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:12.365065 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.365034 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:12.517571 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.517490 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:55:12.518027 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.517987 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" containerID="cri-o://250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66" gracePeriod=30 Jul 10 00:55:12.547467 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.547418 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:12.667503 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.667468 2572 patch_prober.go:28] interesting pod/e2e-api-b-08071d-openshift-default-c98db8c45-sf896 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 00:55:12.667893 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.667522 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 00:55:12.769358 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.769260 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:12.940978 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:12.940360 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:13.355234 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.355201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:13.478197 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.478162 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:13.578940 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.578902 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:13.667395 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.667301 2572 patch_prober.go:28] interesting pod/e2e-api-b-08071d-openshift-default-c98db8c45-sf896 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 10 00:55:13.667395 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.667366 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 10 00:55:13.737794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:13.737761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:14.671302 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:14.671264 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:14.671672 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:14.671533 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:14.672134 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:14.672115 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:17.797325 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.797268 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:17.807348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807329 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807434 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807358 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807434 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807389 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807434 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807413 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkvc5\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807452 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807480 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807516 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807547 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807757 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807571 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs\") pod \"e899e0c5-0029-4485-9c12-08d4f1a82059\" (UID: \"e899e0c5-0029-4485-9c12-08d4f1a82059\") " Jul 10 00:55:17.807757 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807616 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:17.807859 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807763 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:55:17.807921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807874 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:17.807921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807894 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.807921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.807913 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e899e0c5-0029-4485-9c12-08d4f1a82059-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.808110 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.808069 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:17.808634 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.808605 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data" (OuterVolumeSpecName: "istio-data") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:17.809664 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.809635 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:17.809787 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.809768 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:55:17.810002 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.809975 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token" (OuterVolumeSpecName: "istio-token") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:55:17.810086 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.810030 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5" (OuterVolumeSpecName: "kube-api-access-pkvc5") pod "e899e0c5-0029-4485-9c12-08d4f1a82059" (UID: "e899e0c5-0029-4485-9c12-08d4f1a82059"). InnerVolumeSpecName "kube-api-access-pkvc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:55:17.909140 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909107 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909140 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909133 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkvc5\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-kube-api-access-pkvc5\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909140 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909143 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909151 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909159 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e899e0c5-0029-4485-9c12-08d4f1a82059-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909174 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:17.909367 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:17.909182 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e899e0c5-0029-4485-9c12-08d4f1a82059-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:18.297552 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.297514 2572 generic.go:356] "Generic (PLEG): container finished" podID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerID="250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66" exitCode=0 Jul 10 00:55:18.297552 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.297554 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" Jul 10 00:55:18.297834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.297593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerDied","Data":"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66"} Jul 10 00:55:18.297834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.297659 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk" event={"ID":"e899e0c5-0029-4485-9c12-08d4f1a82059","Type":"ContainerDied","Data":"4c1a5de2b5c3cab9fd71520bed251ca33692e4725fce835987877215fd16a304"} Jul 10 00:55:18.297834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.297690 2572 scope.go:117] "RemoveContainer" containerID="250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66" Jul 10 00:55:18.307860 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.307846 2572 scope.go:117] "RemoveContainer" containerID="4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d" Jul 10 00:55:18.317088 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.317071 2572 scope.go:117] "RemoveContainer" containerID="250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66" Jul 10 00:55:18.317363 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:55:18.317345 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66\": container with ID starting with 250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66 not found: ID does not exist" containerID="250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66" Jul 10 00:55:18.317415 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.317374 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66"} err="failed to get container status \"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66\": rpc error: code = NotFound desc = could not find container \"250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66\": container with ID starting with 250c58f6d243e75393d58044bdd356ac00ee7f81af95bb97a52be15d72847f66 not found: ID does not exist" Jul 10 00:55:18.317415 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.317396 2572 scope.go:117] "RemoveContainer" containerID="4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d" Jul 10 00:55:18.317621 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:55:18.317606 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d\": container with ID starting with 4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d not found: ID does not exist" containerID="4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d" Jul 10 00:55:18.317670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.317625 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d"} err="failed to get container status \"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d\": rpc error: code = NotFound desc = could not find container \"4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d\": container with ID starting with 4e32519b4c4ede9657f2c2e619cb33513e4b990b76bbd2155dd8a53767b1613d not found: ID does not exist" Jul 10 00:55:18.319116 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.319097 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:55:18.322699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.322681 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3f46d3-openshift-default-664854b9bc-w5jnk"] Jul 10 00:55:18.514618 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:18.514583 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" path="/var/lib/kubelet/pods/e899e0c5-0029-4485-9c12-08d4f1a82059/volumes" Jul 10 00:55:19.107266 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.107224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:19.172772 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.172735 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:19.357801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.357709 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:55:19.358028 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.358003 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" containerID="cri-o://c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004" gracePeriod=30 Jul 10 00:55:19.380751 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.380719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:19.625484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.625402 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:19.978022 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:19.977924 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:20.271072 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271038 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 00:55:20.271634 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271619 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:55:20.271683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271639 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa177a19-962d-4b7b-bcb2-f2bc3eb1cfc5" containerName="istio-proxy" Jul 10 00:55:20.271683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271663 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:20.271683 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271672 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:20.271797 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271785 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:20.271831 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.271804 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:20.276892 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.276872 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.279078 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.279060 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-df95cf-openshift-default-dockercfg-bhc8l\"" Jul 10 00:55:20.285342 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.285320 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 00:55:20.298064 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.298036 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:20.332870 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.332836 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.332891 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333005 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333031 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333057 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333052 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333241 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333076 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333241 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333241 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.333383 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.333279 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tbrt\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434348 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tbrt\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434452 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434511 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434526 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434529 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434886 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434546 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434886 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434784 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.434886 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.434796 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.435098 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.435077 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.435187 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.435161 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.435238 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.435221 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.437272 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.437250 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.437327 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.437295 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.442840 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.442812 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.443389 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.443372 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tbrt\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt\") pod \"e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.459184 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.459160 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:20.589063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.588982 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:20.728786 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.728755 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 00:55:20.729909 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:55:20.729881 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7cf77d5_02aa_4154_9a92_d16c13c10fc5.slice/crio-09e7c8db85d3790a48e06bf074409089780cbf8ada69a39b5db5f6d6489a2019 WatchSource:0}: Error finding container 09e7c8db85d3790a48e06bf074409089780cbf8ada69a39b5db5f6d6489a2019: Status 404 returned error can't find the container with id 09e7c8db85d3790a48e06bf074409089780cbf8ada69a39b5db5f6d6489a2019 Jul 10 00:55:20.732253 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.732217 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:20.732353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.732300 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:20.732353 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:20.732345 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:21.319333 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:21.319288 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerStarted","Data":"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c"} Jul 10 00:55:21.319333 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:21.319340 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerStarted","Data":"09e7c8db85d3790a48e06bf074409089780cbf8ada69a39b5db5f6d6489a2019"} Jul 10 00:55:21.339796 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:21.339751 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podStartSLOduration=1.33973079 podStartE2EDuration="1.33973079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:55:21.338727805 +0000 UTC m=+2293.365206509" watchObservedRunningTime="2026-07-10 00:55:21.33973079 +0000 UTC m=+2293.366209494" Jul 10 00:55:21.589638 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:21.589548 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:21.594520 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:21.594495 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:22.324862 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:22.324828 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:22.325843 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:22.325824 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:23.265699 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.265665 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:23.355397 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.355367 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:23.420689 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.420658 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:23.521999 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.521898 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:23.810079 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.809994 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:23.911370 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:23.911337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:24.611717 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.611693 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:24.673983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.673897 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.673983 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.673934 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674185 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.673986 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674185 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674020 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm68r\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674185 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674041 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674185 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674135 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674185 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674172 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674231 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674258 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data\") pod \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\" (UID: \"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43\") " Jul 10 00:55:24.674445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674340 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:24.674445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674390 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:55:24.674445 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674414 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:24.674691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674602 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:24.674691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674637 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.674691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674656 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.674691 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.674672 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.675527 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.675497 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:24.676652 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.676624 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 00:55:24.676652 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.676637 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:55:24.676847 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.676828 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r" (OuterVolumeSpecName: "kube-api-access-wm68r") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "kube-api-access-wm68r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:55:24.676914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.676902 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" (UID: "ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:55:24.775540 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775507 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.775540 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775536 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.775540 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775544 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.775766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775554 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm68r\" (UniqueName: \"kubernetes.io/projected/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-kube-api-access-wm68r\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.775766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775563 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:24.775766 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:24.775572 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 00:55:25.344645 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.344610 2572 generic.go:356] "Generic (PLEG): container finished" podID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerID="c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004" exitCode=0 Jul 10 00:55:25.344799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.344655 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" Jul 10 00:55:25.344799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.344687 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerDied","Data":"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004"} Jul 10 00:55:25.344799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.344739 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896" event={"ID":"ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43","Type":"ContainerDied","Data":"948a01fc1be4633a8b6511980d09f1c6300c3a3d5c4fe0c0cf6170498153c59b"} Jul 10 00:55:25.344799 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.344760 2572 scope.go:117] "RemoveContainer" containerID="c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004" Jul 10 00:55:25.355231 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.355215 2572 scope.go:117] "RemoveContainer" containerID="c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0" Jul 10 00:55:25.363671 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.363652 2572 scope.go:117] "RemoveContainer" containerID="c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004" Jul 10 00:55:25.364004 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:55:25.363972 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004\": container with ID starting with c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004 not found: ID does not exist" containerID="c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004" Jul 10 00:55:25.364124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.364015 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004"} err="failed to get container status \"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004\": rpc error: code = NotFound desc = could not find container \"c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004\": container with ID starting with c5fb9fda09574947c37c2fb69b3bb9839c3bb21121ad9c0fc8cd240c154d4004 not found: ID does not exist" Jul 10 00:55:25.364124 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.364038 2572 scope.go:117] "RemoveContainer" containerID="c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0" Jul 10 00:55:25.364619 ip-10-0-142-23 kubenswrapper[2572]: E0710 00:55:25.364591 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0\": container with ID starting with c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0 not found: ID does not exist" containerID="c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0" Jul 10 00:55:25.364710 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.364628 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0"} err="failed to get container status \"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0\": rpc error: code = NotFound desc = could not find container \"c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0\": container with ID starting with c7552d57c23486f5ea653ddad932f43a7e894b06821348ae4a4323e36f2234b0 not found: ID does not exist" Jul 10 00:55:25.367288 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.367270 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:55:25.372991 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:25.372971 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-08071d-openshift-default-c98db8c45-sf896"] Jul 10 00:55:26.513901 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:26.513869 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" path="/var/lib/kubelet/pods/ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43/volumes" Jul 10 00:55:27.797996 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:27.797935 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:28.142981 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.142858 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:28.433665 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.433584 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 00:55:28.433990 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.433977 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.434040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.433991 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.434040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434010 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.434040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434016 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.434040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434027 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:28.434040 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434032 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e899e0c5-0029-4485-9c12-08d4f1a82059" containerName="istio-proxy" Jul 10 00:55:28.434186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434104 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.434186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.434116 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad6f373c-fc14-4c7b-80d6-fa6dd43d6f43" containerName="istio-proxy" Jul 10 00:55:28.438597 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.438575 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.440930 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.440909 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-82df1b-openshift-default-dockercfg-84dxk\"" Jul 10 00:55:28.448163 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.448045 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 00:55:28.454495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.454465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:28.510116 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510078 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510126 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510159 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510245 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510301 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510285 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510358 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwtj\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510452 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.510516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.510496 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.556543 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.556512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:28.611352 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611359 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611508 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwtj\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611566 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611905 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611602 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.611905 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.611767 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.612339 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.612311 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.612492 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.612403 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.612613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.612518 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.612745 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.612719 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.614130 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.614108 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.614375 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.614351 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.621267 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.621240 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.621416 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.621399 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwtj\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj\") pod \"e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.750548 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.750506 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:28.889839 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.889807 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 00:55:28.890862 ip-10-0-142-23 kubenswrapper[2572]: W0710 00:55:28.890834 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c714d86_63a5_40c4_a8e5_053df9b5f5d7.slice/crio-14abb2ed0f4b28d59a13352373e8a78cd5fda7ce0567f7604f76865de2cf3977 WatchSource:0}: Error finding container 14abb2ed0f4b28d59a13352373e8a78cd5fda7ce0567f7604f76865de2cf3977: Status 404 returned error can't find the container with id 14abb2ed0f4b28d59a13352373e8a78cd5fda7ce0567f7604f76865de2cf3977 Jul 10 00:55:28.893063 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.893029 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:28.893178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.893093 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:28.893178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:28.893121 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:29.371704 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:29.371667 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerStarted","Data":"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88"} Jul 10 00:55:29.371917 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:29.371731 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerStarted","Data":"14abb2ed0f4b28d59a13352373e8a78cd5fda7ce0567f7604f76865de2cf3977"} Jul 10 00:55:29.393482 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:29.393426 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podStartSLOduration=1.393411093 podStartE2EDuration="1.393411093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:55:29.390219706 +0000 UTC m=+2301.416698439" watchObservedRunningTime="2026-07-10 00:55:29.393411093 +0000 UTC m=+2301.419889797" Jul 10 00:55:29.751477 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:29.751449 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:29.756171 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:29.756148 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:30.378488 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:30.378458 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:30.379484 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:30.379466 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:31.301779 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:31.301745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:31.400427 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:31.400377 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:31.556767 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:31.556670 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:31.845292 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:31.845205 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:32.257914 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:32.257883 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:32.357451 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:32.357417 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:35.844625 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:35.844582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:38.366093 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:38.366058 2572 patch_prober.go:28] interesting pod/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 00:55:38.366495 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:38.366156 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 00:55:40.454386 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.454350 2572 generic.go:356] "Generic (PLEG): container finished" podID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerID="1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c" exitCode=0 Jul 10 00:55:40.454835 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.454410 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerDied","Data":"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c"} Jul 10 00:55:40.454835 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.454771 2572 scope.go:117] "RemoveContainer" containerID="1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c" Jul 10 00:55:40.455734 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.455705 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:40.455794 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.455780 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:40.455830 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:40.455816 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:41.470031 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:41.469243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerStarted","Data":"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8"} Jul 10 00:55:41.589242 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:41.589207 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:41.589424 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:41.589393 2572 patch_prober.go:28] interesting pod/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 00:55:41.589493 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:41.589443 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 00:55:42.589396 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:42.589359 2572 patch_prober.go:28] interesting pod/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 00:55:42.589804 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:42.589410 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 00:55:43.589932 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:43.589892 2572 patch_prober.go:28] interesting pod/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 00:55:43.590307 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:43.590032 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 00:55:44.506824 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:44.506789 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:44.589670 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:44.589634 2572 patch_prober.go:28] interesting pod/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 10 00:55:44.589842 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:44.589702 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 10 00:55:44.607186 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:44.607159 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:45.593616 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:45.593586 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:45.593934 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:45.593897 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:45.594696 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:45.594676 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 00:55:46.420117 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:46.420071 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 00:55:46.420606 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:46.420134 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 00:55:47.505681 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.505646 2572 generic.go:356] "Generic (PLEG): container finished" podID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerID="204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88" exitCode=0 Jul 10 00:55:47.506178 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.505727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerDied","Data":"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88"} Jul 10 00:55:47.506264 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.506248 2572 scope.go:117] "RemoveContainer" containerID="204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88" Jul 10 00:55:47.507200 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.507171 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:47.507303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.507243 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:47.507303 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:47.507277 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:55:48.515010 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:48.514978 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerStarted","Data":"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c"} Jul 10 00:55:48.751360 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:48.751320 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:48.751564 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:48.751539 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 00:55:48.751613 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:48.751591 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 00:55:49.751762 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:49.751727 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 00:55:49.752196 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:49.751792 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 00:55:50.751834 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:50.751795 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 00:55:50.752354 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:50.751873 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 00:55:51.751814 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:51.751777 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 00:55:51.752030 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:51.751838 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 00:55:52.751201 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:52.751158 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 10 00:55:52.751505 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:52.751222 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 10 00:55:53.737935 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:53.737895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:53.754530 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:53.754505 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:53.754801 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:53.754780 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:53.755473 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:53.755454 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 00:55:53.813437 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:53.813401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:55:56.607188 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:55:56.607157 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:56:07.559569 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:56:07.559536 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:56:22.867912 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:56:22.867879 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:56:33.409566 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:56:33.409530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:56:39.491167 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:56:39.491131 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:56:39.592516 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:56:39.592475 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:57:08.685869 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:57:08.685841 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:57:08.693921 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:57:08.693901 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 00:58:19.099288 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:58:19.099245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:58:19.886317 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:58:19.886272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:58:31.386016 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:58:31.385984 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:08.689152 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:08.688985 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:30.042920 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:30.042885 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:30.144732 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:30.144697 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:44.846363 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:44.846317 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:49.582592 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:49.582553 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:57.535885 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:57.535849 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 00:59:58.403894 ip-10-0-142-23 kubenswrapper[2572]: I0710 00:59:58.403863 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:00.134073 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.134026 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:00:00.138182 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.138163 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:00:00.138291 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.138273 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" Jul 10 01:00:00.140467 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.140449 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9xsl9\"" Jul 10 01:00:00.142096 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.142073 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:00:00.142438 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.142192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" Jul 10 01:00:00.145928 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.145909 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:00:00.146121 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.146002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" Jul 10 01:00:00.147749 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.147731 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:00:00.151617 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.151598 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" Jul 10 01:00:00.167636 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.167614 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:00:00.170765 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.170742 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" Jul 10 01:00:00.203895 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.203861 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" Jul 10 01:00:00.339255 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.335146 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:00:00.339255 ip-10-0-142-23 kubenswrapper[2572]: W0710 01:00:00.336845 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850da394_fbb9_4ab7_8780_27a7dfdb21cb.slice/crio-be1448eb5ac66bf394dcdda3c0ef6a9b354f7187801d5508198a25cbb58343f7 WatchSource:0}: Error finding container be1448eb5ac66bf394dcdda3c0ef6a9b354f7187801d5508198a25cbb58343f7: Status 404 returned error can't find the container with id be1448eb5ac66bf394dcdda3c0ef6a9b354f7187801d5508198a25cbb58343f7 Jul 10 01:00:00.364590 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.364560 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:00:00.368508 ip-10-0-142-23 kubenswrapper[2572]: W0710 01:00:00.366552 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55379a1_3241_4cbf_b470_8592a70645ca.slice/crio-88a156fee2fb4d8a5a13708e37d8de0319549a7558cf66511fe7dc961de3d975 WatchSource:0}: Error finding container 88a156fee2fb4d8a5a13708e37d8de0319549a7558cf66511fe7dc961de3d975: Status 404 returned error can't find the container with id 88a156fee2fb4d8a5a13708e37d8de0319549a7558cf66511fe7dc961de3d975 Jul 10 01:00:00.390703 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.390678 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:00:00.399139 ip-10-0-142-23 kubenswrapper[2572]: W0710 01:00:00.399107 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f716aa5_d582_4bb1_8026_6ad0ad948115.slice/crio-8c1330ac9ab90573c613625656de9248ca2534706f65c5d4e55186a675e065ef WatchSource:0}: Error finding container 8c1330ac9ab90573c613625656de9248ca2534706f65c5d4e55186a675e065ef: Status 404 returned error can't find the container with id 8c1330ac9ab90573c613625656de9248ca2534706f65c5d4e55186a675e065ef Jul 10 01:00:00.971215 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.971101 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" event={"ID":"1f716aa5-d582-4bb1-8026-6ad0ad948115","Type":"ContainerStarted","Data":"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652"} Jul 10 01:00:00.971215 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.971192 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" event={"ID":"1f716aa5-d582-4bb1-8026-6ad0ad948115","Type":"ContainerStarted","Data":"8c1330ac9ab90573c613625656de9248ca2534706f65c5d4e55186a675e065ef"} Jul 10 01:00:00.973294 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.973260 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" event={"ID":"a55379a1-3241-4cbf-b470-8592a70645ca","Type":"ContainerStarted","Data":"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d"} Jul 10 01:00:00.973499 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.973475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" event={"ID":"a55379a1-3241-4cbf-b470-8592a70645ca","Type":"ContainerStarted","Data":"88a156fee2fb4d8a5a13708e37d8de0319549a7558cf66511fe7dc961de3d975"} Jul 10 01:00:00.974914 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.974892 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" event={"ID":"850da394-fbb9-4ab7-8780-27a7dfdb21cb","Type":"ContainerStarted","Data":"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b"} Jul 10 01:00:00.975019 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.974934 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" event={"ID":"850da394-fbb9-4ab7-8780-27a7dfdb21cb","Type":"ContainerStarted","Data":"be1448eb5ac66bf394dcdda3c0ef6a9b354f7187801d5508198a25cbb58343f7"} Jul 10 01:00:00.986577 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:00.986430 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" podStartSLOduration=0.986411907 podStartE2EDuration="986.411907ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:00.985923257 +0000 UTC m=+2573.012401962" watchObservedRunningTime="2026-07-10 01:00:00.986411907 +0000 UTC m=+2573.012890591" Jul 10 01:00:01.000103 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:01.000063 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" podStartSLOduration=1.000047825 podStartE2EDuration="1.000047825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:00.998297027 +0000 UTC m=+2573.024775732" watchObservedRunningTime="2026-07-10 01:00:01.000047825 +0000 UTC m=+2573.026526554" Jul 10 01:00:01.011404 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:01.011363 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" podStartSLOduration=1.011349894 podStartE2EDuration="1.011349894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:00:01.010148923 +0000 UTC m=+2573.036627627" watchObservedRunningTime="2026-07-10 01:00:01.011349894 +0000 UTC m=+2573.037828643" Jul 10 01:00:14.069104 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:14.069069 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:45.223139 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:45.223086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:45.323922 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:45.323877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:54.434818 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:54.434772 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:54.540880 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:54.540851 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:00:58.173335 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:00:58.173295 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:01:00.012614 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.012568 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:01:00.013092 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.012845 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" podUID="1f716aa5-d582-4bb1-8026-6ad0ad948115" containerName="cleanup" containerID="cri-o://48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652" gracePeriod=30 Jul 10 01:01:00.015205 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.015181 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:01:00.015422 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.015385 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" podUID="a55379a1-3241-4cbf-b470-8592a70645ca" containerName="cleanup" containerID="cri-o://fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d" gracePeriod=30 Jul 10 01:01:00.018119 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.018093 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:01:00.018335 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:00.018295 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" podUID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" containerName="cleanup" containerID="cri-o://cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b" gracePeriod=30 Jul 10 01:01:30.227775 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.227743 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" Jul 10 01:01:30.266054 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.266028 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" Jul 10 01:01:30.269574 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.269521 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" Jul 10 01:01:30.495266 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.495228 2572 generic.go:356] "Generic (PLEG): container finished" podID="a55379a1-3241-4cbf-b470-8592a70645ca" containerID="fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d" exitCode=137 Jul 10 01:01:30.495436 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.495284 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" Jul 10 01:01:30.495436 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.495325 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" event={"ID":"a55379a1-3241-4cbf-b470-8592a70645ca","Type":"ContainerDied","Data":"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d"} Jul 10 01:01:30.495436 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.495401 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9" event={"ID":"a55379a1-3241-4cbf-b470-8592a70645ca","Type":"ContainerDied","Data":"88a156fee2fb4d8a5a13708e37d8de0319549a7558cf66511fe7dc961de3d975"} Jul 10 01:01:30.495436 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.495433 2572 scope.go:117] "RemoveContainer" containerID="fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d" Jul 10 01:01:30.496579 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.496558 2572 generic.go:356] "Generic (PLEG): container finished" podID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" containerID="cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b" exitCode=137 Jul 10 01:01:30.496669 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.496599 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" Jul 10 01:01:30.496669 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.496645 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" event={"ID":"850da394-fbb9-4ab7-8780-27a7dfdb21cb","Type":"ContainerDied","Data":"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b"} Jul 10 01:01:30.496763 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.496682 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln" event={"ID":"850da394-fbb9-4ab7-8780-27a7dfdb21cb","Type":"ContainerDied","Data":"be1448eb5ac66bf394dcdda3c0ef6a9b354f7187801d5508198a25cbb58343f7"} Jul 10 01:01:30.498150 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.498129 2572 generic.go:356] "Generic (PLEG): container finished" podID="1f716aa5-d582-4bb1-8026-6ad0ad948115" containerID="48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652" exitCode=137 Jul 10 01:01:30.498241 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.498181 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" Jul 10 01:01:30.498292 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.498255 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" event={"ID":"1f716aa5-d582-4bb1-8026-6ad0ad948115","Type":"ContainerDied","Data":"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652"} Jul 10 01:01:30.498292 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.498288 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt" event={"ID":"1f716aa5-d582-4bb1-8026-6ad0ad948115","Type":"ContainerDied","Data":"8c1330ac9ab90573c613625656de9248ca2534706f65c5d4e55186a675e065ef"} Jul 10 01:01:30.510342 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.510312 2572 scope.go:117] "RemoveContainer" containerID="fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d" Jul 10 01:01:30.510653 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:01:30.510629 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d\": container with ID starting with fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d not found: ID does not exist" containerID="fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d" Jul 10 01:01:30.510734 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.510667 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d"} err="failed to get container status \"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d\": rpc error: code = NotFound desc = could not find container \"fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d\": container with ID starting with fe75c230aed527b5418230a4a82bf7e08f6709bc12d1c27e1fadd7fdf985946d not found: ID does not exist" Jul 10 01:01:30.510734 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.510688 2572 scope.go:117] "RemoveContainer" containerID="cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b" Jul 10 01:01:30.522207 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.522184 2572 scope.go:117] "RemoveContainer" containerID="cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b" Jul 10 01:01:30.522444 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:01:30.522428 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b\": container with ID starting with cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b not found: ID does not exist" containerID="cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b" Jul 10 01:01:30.522503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.522456 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b"} err="failed to get container status \"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b\": rpc error: code = NotFound desc = could not find container \"cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b\": container with ID starting with cac2fe8e7f5ac753b2bd3967d90d095a9900d5bb9fd71d1a9a13733930bbf43b not found: ID does not exist" Jul 10 01:01:30.522503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.522477 2572 scope.go:117] "RemoveContainer" containerID="48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652" Jul 10 01:01:30.528856 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.528834 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:01:30.531248 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.531226 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-82df1b-29727420-7qxt9"] Jul 10 01:01:30.531798 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.531785 2572 scope.go:117] "RemoveContainer" containerID="48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652" Jul 10 01:01:30.532112 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:01:30.532092 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652\": container with ID starting with 48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652 not found: ID does not exist" containerID="48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652" Jul 10 01:01:30.532174 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.532129 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652"} err="failed to get container status \"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652\": rpc error: code = NotFound desc = could not find container \"48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652\": container with ID starting with 48fedd96e63a12a1b6097c420e25c63a8a53be431b72d6564b48f49860d88652 not found: ID does not exist" Jul 10 01:01:30.540179 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.540158 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:01:30.542337 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.542318 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-df95cf-29727420-tq6jt"] Jul 10 01:01:30.551180 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.551158 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:01:30.555277 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:30.555257 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727420-9fgln"] Jul 10 01:01:32.517101 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:32.517070 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f716aa5-d582-4bb1-8026-6ad0ad948115" path="/var/lib/kubelet/pods/1f716aa5-d582-4bb1-8026-6ad0ad948115/volumes" Jul 10 01:01:32.517481 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:32.517334 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" path="/var/lib/kubelet/pods/850da394-fbb9-4ab7-8780-27a7dfdb21cb/volumes" Jul 10 01:01:32.517535 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:32.517522 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a55379a1-3241-4cbf-b470-8592a70645ca" path="/var/lib/kubelet/pods/a55379a1-3241-4cbf-b470-8592a70645ca/volumes" Jul 10 01:01:48.873694 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:48.873655 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:01:50.181595 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:01:50.181562 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:08.727370 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:08.727291 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 01:02:08.736405 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:08.736381 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-prt82_5f623af4-76b4-464c-96ef-5e17791307a8/ovn-acl-logging/0.log" Jul 10 01:02:09.248410 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:09.248370 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:09.740002 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:09.739972 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:09.840886 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:09.840849 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:09.992782 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:09.992693 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:15.609466 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:15.609434 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:15.672448 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:15.672417 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:15.921883 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:15.921796 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 01:02:15.922100 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:15.922021 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" containerID="cri-o://416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8" gracePeriod=30 Jul 10 01:02:15.960704 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:15.960673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:16.141408 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:16.141372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:16.324634 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:16.324590 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:16.739158 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:16.739061 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:16.839164 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:16.839125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:16.993169 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:16.993073 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:21.184048 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.184025 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 01:02:21.262514 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262483 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262686 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262531 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262686 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262565 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262686 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262610 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tbrt\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262686 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262718 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262741 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262773 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.262854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.262818 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket\") pod \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\" (UID: \"c7cf77d5-02aa-4154-9a92-d16c13c10fc5\") " Jul 10 01:02:21.263150 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263031 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:21.263150 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263052 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:21.263150 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263128 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:02:21.263305 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263236 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:21.263305 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263292 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.263406 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263312 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.263406 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263327 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.263889 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.263870 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:21.265420 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.265396 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 01:02:21.265566 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.265547 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt" (OuterVolumeSpecName: "kube-api-access-8tbrt") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "kube-api-access-8tbrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:02:21.265628 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.265585 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:21.265668 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.265612 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7cf77d5-02aa-4154-9a92-d16c13c10fc5" (UID: "c7cf77d5-02aa-4154-9a92-d16c13c10fc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:02:21.364688 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364637 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.364688 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364679 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.364688 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364690 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.364688 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364697 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.364688 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364707 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tbrt\" (UniqueName: \"kubernetes.io/projected/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-kube-api-access-8tbrt\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.365074 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.364715 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7cf77d5-02aa-4154-9a92-d16c13c10fc5-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:21.805758 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.805724 2572 generic.go:356] "Generic (PLEG): container finished" podID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerID="416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8" exitCode=0 Jul 10 01:02:21.805931 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.805772 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" Jul 10 01:02:21.805931 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.805806 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerDied","Data":"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8"} Jul 10 01:02:21.805931 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.805859 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs" event={"ID":"c7cf77d5-02aa-4154-9a92-d16c13c10fc5","Type":"ContainerDied","Data":"09e7c8db85d3790a48e06bf074409089780cbf8ada69a39b5db5f6d6489a2019"} Jul 10 01:02:21.805931 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.805880 2572 scope.go:117] "RemoveContainer" containerID="416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8" Jul 10 01:02:21.816205 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.816183 2572 scope.go:117] "RemoveContainer" containerID="1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c" Jul 10 01:02:21.824709 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.824686 2572 scope.go:117] "RemoveContainer" containerID="416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8" Jul 10 01:02:21.825070 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:02:21.825008 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8\": container with ID starting with 416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8 not found: ID does not exist" containerID="416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8" Jul 10 01:02:21.825070 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.825051 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8"} err="failed to get container status \"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8\": rpc error: code = NotFound desc = could not find container \"416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8\": container with ID starting with 416a6ad59803ca72a5eb4223ae76b64f323c78dffbbf0cafa51a46357898c1f8 not found: ID does not exist" Jul 10 01:02:21.825070 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.825072 2572 scope.go:117] "RemoveContainer" containerID="1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c" Jul 10 01:02:21.825412 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:02:21.825390 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c\": container with ID starting with 1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c not found: ID does not exist" containerID="1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c" Jul 10 01:02:21.825495 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.825417 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c"} err="failed to get container status \"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c\": rpc error: code = NotFound desc = could not find container \"1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c\": container with ID starting with 1d92d7ace5b1bf9a069654e1a44ec877230d4aaef4a5f09a823abe23b07ad48c not found: ID does not exist" Jul 10 01:02:21.827703 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.827684 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 01:02:21.832868 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:21.832847 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-df95cf-openshift-default-57d47954c5-sfdvs"] Jul 10 01:02:22.513345 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.513314 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" path="/var/lib/kubelet/pods/c7cf77d5-02aa-4154-9a92-d16c13c10fc5/volumes" Jul 10 01:02:22.638934 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.638899 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:22.739211 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.739167 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:22.891581 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.891505 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 01:02:22.891735 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.891714 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" containerID="cri-o://c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c" gracePeriod=30 Jul 10 01:02:22.903588 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:22.903545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:23.142293 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:23.142200 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:23.757132 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:23.757098 2572 patch_prober.go:28] interesting pod/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 01:02:23.757529 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:23.757163 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 01:02:28.156329 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.156273 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 01:02:28.335611 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335574 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.335611 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335611 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.335866 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335692 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzwtj\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.335866 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335710 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.335866 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335743 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.336080 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335890 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.336080 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335907 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:28.336080 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.335934 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.336080 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336036 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.336080 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336072 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token\") pod \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\" (UID: \"6c714d86-63a5-40c4-a8e5-053df9b5f5d7\") " Jul 10 01:02:28.336322 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336079 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:28.336322 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336168 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:28.336322 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336200 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:02:28.336518 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336415 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.336518 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336432 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-workload-certs\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.336518 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336447 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istiod-ca-cert\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.336518 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.336460 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-credential-socket\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.337058 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.337031 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:28.338221 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.338194 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj" (OuterVolumeSpecName: "kube-api-access-gzwtj") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "kube-api-access-gzwtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:02:28.338339 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.338231 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 01:02:28.338405 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.338340 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:02:28.338405 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.338366 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c714d86-63a5-40c4-a8e5-053df9b5f5d7" (UID: "6c714d86-63a5-40c4-a8e5-053df9b5f5d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 01:02:28.437857 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.437765 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-data\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.437857 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.437796 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzwtj\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-kube-api-access-gzwtj\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.437857 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.437811 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-podinfo\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.437857 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.437823 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-envoy\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.437857 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.437833 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c714d86-63a5-40c4-a8e5-053df9b5f5d7-istio-token\") on node \"ip-10-0-142-23.ec2.internal\" DevicePath \"\"" Jul 10 01:02:28.742220 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.742127 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gzbxk_c2eaf9cc-d308-43a0-8cca-35f16e605aeb/manager/0.log" Jul 10 01:02:28.849572 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.849538 2572 generic.go:356] "Generic (PLEG): container finished" podID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerID="c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c" exitCode=0 Jul 10 01:02:28.849745 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.849589 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" Jul 10 01:02:28.849745 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.849622 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerDied","Data":"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c"} Jul 10 01:02:28.849745 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.849669 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq" event={"ID":"6c714d86-63a5-40c4-a8e5-053df9b5f5d7","Type":"ContainerDied","Data":"14abb2ed0f4b28d59a13352373e8a78cd5fda7ce0567f7604f76865de2cf3977"} Jul 10 01:02:28.849745 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.849691 2572 scope.go:117] "RemoveContainer" containerID="c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c" Jul 10 01:02:28.858802 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.858780 2572 scope.go:117] "RemoveContainer" containerID="204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88" Jul 10 01:02:28.867278 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.867258 2572 scope.go:117] "RemoveContainer" containerID="c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c" Jul 10 01:02:28.867551 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:02:28.867530 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c\": container with ID starting with c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c not found: ID does not exist" containerID="c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c" Jul 10 01:02:28.867640 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.867556 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c"} err="failed to get container status \"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c\": rpc error: code = NotFound desc = could not find container \"c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c\": container with ID starting with c1c5d77451a7eb7735d8163e5a151f7b9790ca1de154dc85802335023ff7027c not found: ID does not exist" Jul 10 01:02:28.867640 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.867578 2572 scope.go:117] "RemoveContainer" containerID="204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88" Jul 10 01:02:28.867869 ip-10-0-142-23 kubenswrapper[2572]: E0710 01:02:28.867848 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88\": container with ID starting with 204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88 not found: ID does not exist" containerID="204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88" Jul 10 01:02:28.867939 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.867878 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88"} err="failed to get container status \"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88\": rpc error: code = NotFound desc = could not find container \"204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88\": container with ID starting with 204e22e1ae34376340c7f37c2c0bff3823a722e3a1fdd7e2f911c1cf3d182d88 not found: ID does not exist" Jul 10 01:02:28.870035 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.870013 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 01:02:28.873086 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.873067 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-82df1b-openshift-default-6bd4dbcb5f-984sq"] Jul 10 01:02:28.893820 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:28.893790 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zl9h7_8dfe28e5-4518-4289-a6a9-f4898c92c92f/manager/0.log" Jul 10 01:02:29.179063 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:29.179031 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qjx67_7a10a9e2-b0d2-4377-a722-90bbf2dd5929/server/0.log" Jul 10 01:02:29.313500 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:29.313462 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-69dn5_76c5e371-465f-48f4-951d-cc695e6eaa9d/manager/0.log" Jul 10 01:02:29.444152 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:29.444073 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d875657f-52h9r_66e5bf26-c5c4-4f07-9d75-9693d63b665c/copy-manifests/0.log" Jul 10 01:02:29.450746 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:29.450717 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d875657f-52h9r_66e5bf26-c5c4-4f07-9d75-9693d63b665c/manager/0.log" Jul 10 01:02:30.514261 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:30.514226 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" path="/var/lib/kubelet/pods/6c714d86-63a5-40c4-a8e5-053df9b5f5d7/volumes" Jul 10 01:02:31.018707 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.018656 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/extract/0.log" Jul 10 01:02:31.024752 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.024730 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/util/0.log" Jul 10 01:02:31.030828 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.030805 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/pull/0.log" Jul 10 01:02:31.150023 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.149996 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/util/0.log" Jul 10 01:02:31.156980 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.156925 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/pull/0.log" Jul 10 01:02:31.163484 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.163459 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/extract/0.log" Jul 10 01:02:31.294057 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.293978 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/pull/0.log" Jul 10 01:02:31.301470 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.301449 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/extract/0.log" Jul 10 01:02:31.307940 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.307918 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/util/0.log" Jul 10 01:02:31.426310 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.426276 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/util/0.log" Jul 10 01:02:31.433281 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.433249 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/pull/0.log" Jul 10 01:02:31.439919 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.439897 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/extract/0.log" Jul 10 01:02:31.700294 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:31.700204 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-n2k5j_b223879f-13ef-4166-b16c-4625b374efdd/manager/0.log" Jul 10 01:02:32.465408 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:32.465374 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wdxxb_8c5692a8-275f-458c-ad74-3ab741c03dce/manager/0.log" Jul 10 01:02:32.978317 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:32.978285 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4l2fh_0fa9c659-33bc-4916-b33f-c1865ee56fe3/discovery/0.log" Jul 10 01:02:33.101421 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:33.101392 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77865f7b8-lz6q4_0721fc09-8f0a-4f00-82e9-9b5fd08171dc/kube-auth-proxy/0.log" Jul 10 01:02:33.715547 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:33.715520 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-678f64b94d-llqjf_fdc51d59-fb65-4059-a734-ceeb6b050aec/router/0.log" Jul 10 01:02:34.077249 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:34.077213 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gdwnd_7eeffbb2-a028-48cb-89ae-a72baf787737/main/0.log" Jul 10 01:02:39.399537 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:39.399503 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:39.460231 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:39.460184 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" limit=1024 Jul 10 01:02:42.037852 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:42.037819 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2c8lv_c2168193-a602-41e7-adc0-7c0a76072865/global-pull-secret-syncer/0.log" Jul 10 01:02:42.164936 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:42.164906 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hbxb_7d84ec5b-9b0b-4e1f-a21e-e8240ba459e9/konnectivity-agent/0.log" Jul 10 01:02:42.308135 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:42.308051 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-23.ec2.internal_b630ef5ac97d738b7ebb71a3cf093891/haproxy/0.log" Jul 10 01:02:46.183623 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.183593 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/extract/0.log" Jul 10 01:02:46.210220 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.210191 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/util/0.log" Jul 10 01:02:46.231941 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.231910 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dmrs7_eb454caa-5774-40bb-899c-7f9979f36f5d/pull/0.log" Jul 10 01:02:46.258664 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.258640 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/extract/0.log" Jul 10 01:02:46.288700 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.288669 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/util/0.log" Jul 10 01:02:46.311045 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.311023 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04vvgj_98f8a1f1-445c-4c1a-86d0-ed3add531597/pull/0.log" Jul 10 01:02:46.336536 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.336509 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/extract/0.log" Jul 10 01:02:46.356801 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.356783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/util/0.log" Jul 10 01:02:46.378279 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.378258 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gcrwp_37c94478-edd5-472e-8d74-7121197d8f75/pull/0.log" Jul 10 01:02:46.402888 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.402867 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/extract/0.log" Jul 10 01:02:46.422682 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.422659 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/util/0.log" Jul 10 01:02:46.443392 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:46.443318 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1bllv4_b3f6a9f1-37c1-444f-9839-051b3d89f3af/pull/0.log" Jul 10 01:02:47.021319 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:47.021293 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-n2k5j_b223879f-13ef-4166-b16c-4625b374efdd/manager/0.log" Jul 10 01:02:47.420664 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:47.420578 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wdxxb_8c5692a8-275f-458c-ad74-3ab741c03dce/manager/0.log" Jul 10 01:02:49.154780 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.154747 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-d4nf2_879948e0-3f63-4ada-b677-1c55541823d3/cluster-monitoring-operator/0.log" Jul 10 01:02:49.201546 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.201507 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xzj7v_1bbfc9fb-45d3-4543-9757-534530eae4bd/kube-state-metrics/0.log" Jul 10 01:02:49.223823 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.223788 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xzj7v_1bbfc9fb-45d3-4543-9757-534530eae4bd/kube-rbac-proxy-main/0.log" Jul 10 01:02:49.244814 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.244789 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-xzj7v_1bbfc9fb-45d3-4543-9757-534530eae4bd/kube-rbac-proxy-self/0.log" Jul 10 01:02:49.332088 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.332050 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27llx_a75c383d-36ec-4b19-84e2-038315bb0e5e/node-exporter/0.log" Jul 10 01:02:49.350794 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.350769 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27llx_a75c383d-36ec-4b19-84e2-038315bb0e5e/kube-rbac-proxy/0.log" Jul 10 01:02:49.371409 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.371389 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27llx_a75c383d-36ec-4b19-84e2-038315bb0e5e/init-textfile/0.log" Jul 10 01:02:49.580316 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.580286 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xmvz5_d6537777-41b8-4112-8f4e-542367b12c15/kube-rbac-proxy-main/0.log" Jul 10 01:02:49.602823 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.602801 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xmvz5_d6537777-41b8-4112-8f4e-542367b12c15/kube-rbac-proxy-self/0.log" Jul 10 01:02:49.625370 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.625349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xmvz5_d6537777-41b8-4112-8f4e-542367b12c15/openshift-state-metrics/0.log" Jul 10 01:02:49.858059 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.857968 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nxpln_22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc/prometheus-operator/0.log" Jul 10 01:02:49.874001 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.873978 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nxpln_22d9d42d-ddb2-4b18-95f8-bd8a13ad4cfc/kube-rbac-proxy/0.log" Jul 10 01:02:49.900425 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:49.900402 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-mg6hg_82768138-6201-43c1-901a-915be3e69f51/prometheus-operator-admission-webhook/0.log" Jul 10 01:02:50.004789 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.004759 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/thanos-query/0.log" Jul 10 01:02:50.024863 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.024841 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/kube-rbac-proxy-web/0.log" Jul 10 01:02:50.044136 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.044112 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/kube-rbac-proxy/0.log" Jul 10 01:02:50.063334 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.063313 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/prom-label-proxy/0.log" Jul 10 01:02:50.082824 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.082782 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/kube-rbac-proxy-rules/0.log" Jul 10 01:02:50.105133 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:50.105108 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7969c5-xfcls_3b1ed516-eb97-426a-b877-fed811c75234/kube-rbac-proxy-metrics/0.log" Jul 10 01:02:51.410335 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:51.410277 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-q4kw4_fe453deb-c767-4f25-8637-46a098957006/networking-console-plugin/0.log" Jul 10 01:02:52.495470 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.495439 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t"] Jul 10 01:02:52.496032 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496011 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496034 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496046 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496053 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496067 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f716aa5-d582-4bb1-8026-6ad0ad948115" containerName="cleanup" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496076 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f716aa5-d582-4bb1-8026-6ad0ad948115" containerName="cleanup" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496107 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" containerName="cleanup" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496116 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" containerName="cleanup" Jul 10 01:02:52.496126 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496125 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a55379a1-3241-4cbf-b470-8592a70645ca" containerName="cleanup" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496133 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a55379a1-3241-4cbf-b470-8592a70645ca" containerName="cleanup" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496249 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="850da394-fbb9-4ab7-8780-27a7dfdb21cb" containerName="cleanup" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496264 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a55379a1-3241-4cbf-b470-8592a70645ca" containerName="cleanup" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496274 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496281 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c714d86-63a5-40c4-a8e5-053df9b5f5d7" containerName="istio-proxy" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496290 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496298 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cf77d5-02aa-4154-9a92-d16c13c10fc5" containerName="istio-proxy" Jul 10 01:02:52.496503 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.496307 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f716aa5-d582-4bb1-8026-6ad0ad948115" containerName="cleanup" Jul 10 01:02:52.499685 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.499664 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.502142 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.502112 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6cvbh\"/\"openshift-service-ca.crt\"" Jul 10 01:02:52.502262 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.502150 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6cvbh\"/\"kube-root-ca.crt\"" Jul 10 01:02:52.502762 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.502732 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6cvbh\"/\"default-dockercfg-njqvb\"" Jul 10 01:02:52.504912 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.504842 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t"] Jul 10 01:02:52.548926 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.548892 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-podres\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.549139 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.548970 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-sys\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.549139 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.549050 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-proc\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.549139 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.549084 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-lib-modules\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.549270 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.549164 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbf69\" (UniqueName: \"kubernetes.io/projected/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-kube-api-access-wbf69\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.553264 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.553225 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c8f4c8746-bcszv_470aea4f-ea26-4452-9183-2f7115ce0355/console/0.log" Jul 10 01:02:52.650402 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650371 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-proc\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650402 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-lib-modules\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650434 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbf69\" (UniqueName: \"kubernetes.io/projected/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-kube-api-access-wbf69\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650476 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-podres\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-sys\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650520 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-proc\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650564 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-lib-modules\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650583 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-sys\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.650658 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.650632 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-podres\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.658536 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.658511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbf69\" (UniqueName: \"kubernetes.io/projected/243b869b-7c48-4fe8-8d2a-fb60e2f87c1c-kube-api-access-wbf69\") pod \"perf-node-gather-daemonset-d9v6t\" (UID: \"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c\") " pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.811661 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.811625 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:52.947567 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.947539 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t"] Jul 10 01:02:52.948966 ip-10-0-142-23 kubenswrapper[2572]: W0710 01:02:52.948926 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod243b869b_7c48_4fe8_8d2a_fb60e2f87c1c.slice/crio-31da5f4cef4d27ad61e3a51ef88bd8d58f23366be22055dc2899ff049aa0556e WatchSource:0}: Error finding container 31da5f4cef4d27ad61e3a51ef88bd8d58f23366be22055dc2899ff049aa0556e: Status 404 returned error can't find the container with id 31da5f4cef4d27ad61e3a51ef88bd8d58f23366be22055dc2899ff049aa0556e Jul 10 01:02:52.982606 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:52.982580 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" event={"ID":"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c","Type":"ContainerStarted","Data":"31da5f4cef4d27ad61e3a51ef88bd8d58f23366be22055dc2899ff049aa0556e"} Jul 10 01:02:53.989630 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:53.989589 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" event={"ID":"243b869b-7c48-4fe8-8d2a-fb60e2f87c1c","Type":"ContainerStarted","Data":"d181dad1a61d9bede7bf09543c87ae294394cbbd628872040b44800a870fca12"} Jul 10 01:02:53.990063 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:53.989660 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:02:54.007334 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:54.007285 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" podStartSLOduration=2.007270036 podStartE2EDuration="2.007270036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:02:54.006441925 +0000 UTC m=+2746.032920642" watchObservedRunningTime="2026-07-10 01:02:54.007270036 +0000 UTC m=+2746.033748740" Jul 10 01:02:54.187471 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:54.187440 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xjzv_f421c76d-3cf5-4d11-bd6c-95ae39675933/dns/0.log" Jul 10 01:02:54.206276 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:54.206248 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xjzv_f421c76d-3cf5-4d11-bd6c-95ae39675933/kube-rbac-proxy/0.log" Jul 10 01:02:54.356611 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:54.356583 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v8t9f_5fe2c04e-788c-49c7-bdb0-40d66299f6f9/dns-node-resolver/0.log" Jul 10 01:02:54.913114 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:54.913086 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dmbpk_96c8aa3a-d36e-42fd-b36f-54a04b7d24d4/node-ca/0.log" Jul 10 01:02:56.057798 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:56.057763 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4l2fh_0fa9c659-33bc-4916-b33f-c1865ee56fe3/discovery/0.log" Jul 10 01:02:56.079854 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:56.079825 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77865f7b8-lz6q4_0721fc09-8f0a-4f00-82e9-9b5fd08171dc/kube-auth-proxy/0.log" Jul 10 01:02:56.254977 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:56.254929 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-678f64b94d-llqjf_fdc51d59-fb65-4059-a734-ceeb6b050aec/router/0.log" Jul 10 01:02:56.926885 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:56.926846 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-npqnr_65c91095-98b9-46b3-8a68-62eeccdab145/serve-healthcheck-canary/0.log" Jul 10 01:02:57.407943 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:57.407911 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7ppbl_d95732d1-a141-48f0-afff-e90eefae17d3/insights-operator/0.log" Jul 10 01:02:57.408486 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:57.408440 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7ppbl_d95732d1-a141-48f0-afff-e90eefae17d3/insights-operator/1.log" Jul 10 01:02:57.492103 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:57.492069 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9z4k_ff07e5ee-efae-401a-8f45-d9546536e6a5/kube-rbac-proxy/0.log" Jul 10 01:02:57.511184 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:57.511159 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9z4k_ff07e5ee-efae-401a-8f45-d9546536e6a5/exporter/0.log" Jul 10 01:02:57.538229 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:57.538203 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k9z4k_ff07e5ee-efae-401a-8f45-d9546536e6a5/extractor/0.log" Jul 10 01:02:59.649347 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:59.649310 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gzbxk_c2eaf9cc-d308-43a0-8cca-35f16e605aeb/manager/0.log" Jul 10 01:02:59.876403 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:02:59.876376 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zl9h7_8dfe28e5-4518-4289-a6a9-f4898c92c92f/manager/0.log" Jul 10 01:03:00.005980 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:00.005939 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6cvbh/perf-node-gather-daemonset-d9v6t" Jul 10 01:03:00.024818 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:00.024784 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qjx67_7a10a9e2-b0d2-4377-a722-90bbf2dd5929/server/0.log" Jul 10 01:03:00.129111 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:00.129076 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-69dn5_76c5e371-465f-48f4-951d-cc695e6eaa9d/manager/0.log" Jul 10 01:03:00.152327 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:00.152298 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d875657f-52h9r_66e5bf26-c5c4-4f07-9d75-9693d63b665c/manager/0.log" Jul 10 01:03:00.171108 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:00.171086 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d875657f-52h9r_66e5bf26-c5c4-4f07-9d75-9693d63b665c/copy-manifests/0.log" Jul 10 01:03:01.725089 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:01.725058 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4hmxk_02eb94e2-e914-4aed-9192-2e7fa6cb2356/openshift-lws-operator/0.log" Jul 10 01:03:02.407248 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:02.407217 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4l2fh_0fa9c659-33bc-4916-b33f-c1865ee56fe3/discovery/0.log" Jul 10 01:03:02.416615 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:02.416585 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77865f7b8-lz6q4_0721fc09-8f0a-4f00-82e9-9b5fd08171dc/kube-auth-proxy/0.log" Jul 10 01:03:02.555615 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:02.555578 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-678f64b94d-llqjf_fdc51d59-fb65-4059-a734-ceeb6b050aec/router/0.log" Jul 10 01:03:03.846733 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:03.846706 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-4l2fh_0fa9c659-33bc-4916-b33f-c1865ee56fe3/discovery/0.log" Jul 10 01:03:03.859691 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:03.859658 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77865f7b8-lz6q4_0721fc09-8f0a-4f00-82e9-9b5fd08171dc/kube-auth-proxy/0.log" Jul 10 01:03:04.013265 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:04.013232 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-678f64b94d-llqjf_fdc51d59-fb65-4059-a734-ceeb6b050aec/router/0.log" Jul 10 01:03:07.226394 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:07.226366 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-kz244_e604290b-7f97-494d-82ad-55d29971c045/kube-storage-version-migrator-operator/1.log" Jul 10 01:03:07.228108 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:07.228084 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-kz244_e604290b-7f97-494d-82ad-55d29971c045/kube-storage-version-migrator-operator/0.log" Jul 10 01:03:08.517210 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.517180 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/kube-multus-additional-cni-plugins/0.log" Jul 10 01:03:08.537296 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.537264 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/egress-router-binary-copy/0.log" Jul 10 01:03:08.556460 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.556440 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/cni-plugins/0.log" Jul 10 01:03:08.576097 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.576070 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/bond-cni-plugin/0.log" Jul 10 01:03:08.596044 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.596021 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/routeoverride-cni/0.log" Jul 10 01:03:08.615312 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.615292 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/whereabouts-cni-bincopy/0.log" Jul 10 01:03:08.634748 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.634725 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wb9hb_45825668-2a70-4327-a7f5-6ad932ab0eb6/whereabouts-cni/0.log" Jul 10 01:03:08.736437 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.736405 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r9kxr_623f5d31-40e4-4687-af26-0c4a575523be/kube-multus/0.log" Jul 10 01:03:08.800671 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.800579 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n7czp_13e20826-95ee-455c-8405-f23540349d1e/network-metrics-daemon/0.log" Jul 10 01:03:08.815687 ip-10-0-142-23 kubenswrapper[2572]: I0710 01:03:08.815657 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n7czp_13e20826-95ee-455c-8405-f23540349d1e/kube-rbac-proxy/0.log"